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Preface 



This paper is one in a series of Papers for Practitioners that was conceived, as its 
name suggests, with a particular audience in mind: the "doers"— in this case, the 
people with day-to-day responsibility for operating programs that provide services 
to low-income individuals and families. The series is intended to bring lessons from 
research to the tasks of designing and implementing such programs, and to provide 
an outlet for information about the practical program management and operational 
issues that often get less attention than bottom-line results in research documents. 

Assessment in the Job Opportunities and Basic Skills Training (JOBS) Program for 
welfare recipients is a topic that fits closely vrith MDRCs vision of Papers for 
Practitioners. It has not been studied systematically, but it is vitally important to 
JOBS participants, adnunistrators, and caseworkers. Assessment philosophies and 
practices set in nrK)tion the service delivery system's response to people who come 
forward, although researchers have devoted more attention to service responses 
themselves and their effects. Assessment has gained prominence recently in the 
employment and training system for low-income adults, spurred in part by 
provisions of the Family Support Act of 1988 that created the JOBS program. 

MDRCs nussion is the identification and rigorous field evaluation of programs 
that promise to improve the life prospects of the disadvantaged. But the research 
projects we carry out are meaningful to practitioners only if the results— both of 
programs that prove successful and those that do not achieve their goals— are known 
and understood by the people who make decisions about which programs and 
policies to adopt. An understanding of the processes inside social programs and 
policies-such as how clients' capabilities, preferences, and needs are assessed— is also 
useful to these decision-makers, but essential to practitioners. We hope this paper 
contributes to that understanding. 

Judith M. Gueron 
President 
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Introduction 



Programs that provide employment-related services, including education and job 
training, almost always start v/ith some form of assessment to determine whether 
the individuals seeking to enroll can benefit from the services offered. In multi- 
component programs such as the Job Opportunities and Basic Skills Training (JOBS) 
Program for welfare recipients, created by the Family Support Act of 1988, 
"assessment" is the name for the process by which eligible individuals with varying 
characteristics are matched to different services ranging in JOBS from short-term job- 
search assistance to four-year college. Although the decisions made in this matching 
process have profound implications for the cost, operation, and effectiveness of 
employment programs, assessment has been infrequently studied. It remains 
mysterious to nnany of the administrators responsible for designing and managing 
programs to achieve specific employment-related goals. Choices about assessment 
practices are often delegated to vocational training specialists or intake staff. 

This paper, intended for administrators of JOBS programs, conveys a simple 
message: Assessment practices will shape your program in important ways, 
because it is through assessment that "who gets what** in JOBS is determined. 
JOBS administrators need to understand their choices about assessment approaches, 
investigate their assessment options, and choose deliberately, based on their goals 
for JOBS and the best available information about how different approaches might 
support or conflict with those goals. 

The paper raises questions that the authors believe JOBS administrators should 
be asking themselves, their staff, and service providers in the program. It also takes 
a view of client assessment in JOBS that runs counter to current trends and thinking 
among service providers. In our contact with JOBS programs during the last two 
years, we have seen an increasing emphasis on formal assessment. There is more 
testing going on, more one-to-one counseling about clients' options for program 
participation, and greater reliance on career exploration as a first step in JOBS. This 
trend is consistent with the JOBS program described in the Family Support Act. But 
it is not witliout costs. In this paper, we give more weight to tlie disadvantages of 
formal assessment approaches than is usual in the literature and in the field at this 
time, so that administrators can assess trade-offs more accurately. 

Our sources include employment and training programs for nonwelfare 
populations as well as welfare employment programs operated under WIN (the 
Work Incentive Program) and WIN Demonstration auspices prior to JOBS. In recent 
years, the government-supported employment and training system as a 
whole-including programs funded through the Job Training Partnership Act of 1982 
OTP A), the national Employment Service, and the military— has devoted considerable 
attention to the development of reliable strategies for identifying the education 
needs of disadvantaged youths and adults and preparing them for employment. 
Now tl^at welfare agency administrators are principally responsible for shaping 
JOBS programs to fit the needs of their clients, tliey need to be familiar with the 
"state of the art" in these related labor supply systems. 
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How To Use This Document 

To make the subtopics and conclusions of this paper accessible to readers 
interested in particular assessment issues, three types of guides to the text are 
provided: 

• Table of Contents: Readers can choose topics of interest by scanning chapter and 
section headings. Section headings are printed in boldface type in the text for 
quick identification. 

• Summaries: At the end of many chapters is a boxed section entitled "Key Issues." 
(The Contents gives page numbers for these sections.) Readers can identify parts 
of the detailed discussion in which they have a special interest by scanning these 
summaries. 

• Major Points: Throughout the text, important lessons about assessment issues are 
printed in boldface type. Readers can scan these to identify the issues most 
relevant to their programs. 

The paper contains nine chapters. Chapter 1 explains why assessment matters 
to JOBS administrators. Chapter 2 suggests a conceptual framework for assessment 
in JOBS and identifies dimensions along which assessment systems can vary. 
Chapters discusses key issues in assessing participants' immediate employability^ 
or job-readiness. Strategies and issues in assessing basic literacy, vocational 
abilities and interests, and social service needs are presented in Chapters 4, 5, and 
6, respectively. 

Chapter 7 describes the experiences of several welfare-to-work programs in 
designing and implementing assessment systems. Chapter 8 reviews operational 
considerations that should inform JOBS administrators' decisions about assessment 
practices, including resource constraints, staffing, the need to tailor assessments to 
special populations, and information-flow issues. The Conclusion reviews trade-offs 
in overall JOBS program design as they relate to assessment. 

Although we discuss some tests and other assessment products, we do not offer 
endorsements or recommendations, since the appropriateness of particular methods 
and materials depends heavily on the character of particular JOBS programs and the 
communities they serve. The assessment choices that JOBS administrators 
make— among them, whether to purchase proprietary assessment products, borrow 
from other programs' materials and approaches, or develop their own assessment 
systems-must be framed by their own programs' objectives and designs, operating 
environments, budget constraints, and target populations. Tliis docunrtent will help 
administrators weigh the possible benefits and drav;backs of the choices they must 
make. 



Chapter 1 



Why Worry About Assessment? 

Under the Family Support Act of 1988 (FSA), states are encouraged to serve a 
broad segment of the welfare caseload in employment programs and to emphasize 
providing education and training opportunities for welfare recipients. As a result 
many states are operating Job Opportunities and Basic Skills Training (JOBS) 
programs established by the act, which are more complex than the welfare-to-work 
programs typical of the 1980s. To aid in the implementation of programs that offer 
multiple services, JOBS requires an assessment of each participant.^ 

With the introduction of the JOBS program, variations in assessment practices 
have new significance. Although the overall purpose of assessment in JOBS is to 
develop an employability plan for each program participant, the statute and the 
federal regulations governing the program's operation are written broadly enough 
to encompass many assessment objectives and practices.^ Neither the statute nor the 
regulations specify particular methodologies, strategies, or instruments, nor do they 
require states to administer tests. They set no time limit for conducting an initial 
assessment, except that applicants for Aid to Families with Dependent Children 
(AFDC) cannot be placed in a job-search activity for more than three weeks before 
having such an assessment and that all initial assessments are supposed to occur 
within a "reasonable" amount of time.^ 

State and local administrators thus have many options. Essentially, they must 
design their own assessment program: choose its form and duration, decide where 
it fits in a sequence of JOBS activities, establish criteria specifying which program 
activities are appropriate for participants with different characteristics, select the 
methods and instruments, and formulate procedures for using the information 
obtained. Unfortunately, there is no consensus in the field about how such decisions 
should be made. 

The choices that JOBS administrators make have important implications for their 
programs, however. As JOBS expands the range of employment-related services 



'Throughout this paper, "participant" is used to refer to welfare recipients who are potential 
participants in JOBS, even though they may not actually attend program activities. 

We use female pronouns to refer to JOBS participants because at present most are female. The 
number of male participants will eventually increase, given FSA requirements that states operate 
cash assistance programs for intact families in which the principal earner is unemployed, and that 
they provide a specific set of employment services for heads of these two-parent households. 

^hey only require states to conduct an initial assessment of a participant's employability by 
gathering information about her: (1) education and child care and other supportive-service needs; 
(2) proficiencies, skills deficiencies, and prior work experience; and (3) family circumstances, which 
may include the needs of her children. See P.L. 100-485, Sec. 201(b), creating Sec. 482(b)(1)(A) of 
the Social Security Act. 

•^See 45 CFR 250.41, the final rule adopted on October 13, 1989, regarding the "initial assessment 
and employability plan," under Subpart E, Operationof State JOBS Program/Program Components, 
and 45 CFR 250.60, the rule regarding the "job search program," under Subpart G, Optional 
Components of State JOBS Programs. 

3 
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available to welfare recipients, states are adopting more complex assessment 
procedures than they have used in the past. The feasibility and cost-effectiveness of 
different assessment approaches have new importance to JOBS administrators under 
these conditions. Their decisions about how much of what kind of assessment to 
provide are doubly important because JOBS is often in competition for funds with 
education, job training, and other employment-related services provided to JOBS 
participants. In addition: 

• Decisions about "who goes where" in JOBS progran\s are a major influence on 
program operations. For example, the services available in many JOBS programs 
include three- to four-week job-club workshops, open-€nded adult basic 
education (ABE) and English as a second language (ESL) courses, and intensive, 
relatively expensive vocational skills training that lasts several months. Decisions 
about which welfare recipients will be assigned to these activities can affect the 
rate at which eligible individuals participate in JOBS, the average length of time 
people spend in the program, how resources are allocated among different 
groups of clients and types of services, completion rates, and total program costs. 

• The methods employed to arrive at decisions about "who goes where"— for 
example, whether assessment is brief or lengthy, made up of one or many steps, 
carried out in one or many locations by one or many agencies— also clearly affect 
program operations. The number of staff members needed for assessment, their 
level of qualifications, the importance of inter-agency cooperation, and client- 
tracking procedures in all these vary. 

• Although it is far too early to have reliable information about the effects of 
different JOBS approaches, assessment decisions that target certain people for 
particular activities may play a key role in determining overall program success. 
If some JOBS comf>onents prove more effective than others for certain 
participants, then accuracy in tl^e assessment process may be critical to the 
effectiveness of the program. 

• The JOBS program emphasizes education as a primary service for participants 
who lack a high school diploma (or its equivalent) and basic literacy skills. The 
program regulations define basic literacy as skills allowing a person to function 
at a level equivalent to at least grade 8.9. A key assessment question is how to 
deternaine participants' basic literacy skill levels with this degree of precision and 
yet avoid wasting time and resources. 

Too Much Versus Too Little 

JOBS administrators face two risks in choosing assessment systems for their 
programs. On one hand, an overly elaborate assessment process could eat up 
resources, delay entry into program components and employment, and reduce 
overall program participation. On the other hand, inadequate assessment 
procedures or inappropriate criteria for deciding "who goes where" could lead 
staff to refer participants to activities that do not succeed in increasing their 
employability and reducing their reliance on welfare. 
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Obscuring the path between these risks is a tliicket of interconnections among 
assessment methods, program operations, and the philosophy, goals, and structure 
of JOBS programs. Research evidence illuminates the problem but does not offer a 
clear solution. For example, the research conducted on welfare employment 
programs operated under the Work Incentive (WIN) or WIN Demonstration 
structure in ti\e 1980s underscores the importance of matching services to clients. The 
interventions studied in the 1980s^which were often a fixed sequence of activities 
beginning with an assisted job search- were found to be more effective for some of 
the welfare recipients required to participate (who were usually single mothers with 
children of school age) than for others. Specifically, most programs resulted in 
positive and sustained earnings gains, on average, for moderately disadvantaged 
welfare recipients who had received assistance before and were reapplying, 
sometimes after employment in jobs with limited earnings. At the same time, there 
was little improvement for either the most job-ready (who often leave welfare for 
woT-k, but would do so even without help) or tor the most disadvantaged.'* 

Together with analyses of welfare caseload dynamics— which demonstrated that 
long-term dependency is concentrated in a relatively small, highly disadvantaged 
group^— these findings constituted an impetus for innovations realized in JOBS. The 
authors of the Family Support Act hoped to improve on the effectiveness of WIN, 
especially for groups likely to remain dependent for many years, by expanding 
program options, adding education as a program response for some groups, 
establishing goals for serving specified target groups (including long-term welfare 
recipients), and introducing assessment as the mechanism for determining which 
services are appropriate for individual welfare recipients. 

While studies of the pre-JOBS generation of welfare-to-work programs clearly 
show diversity within the eligible population and different program effect:, ox. 
different subgroups, they also have important limitations. First, the average findi ^.^i 
for major subgroups— those described as the most job-ready, the moderate-y 
disadvantaged, and the most disadvantaged welfare recipients— masked a great dc.^1 
of variation. For example, the welfare histories of participants, which were oi^e 
the most powerful characteristics correlating with program impact,^ could stili l\* 
overshadowed in any single individual by other characteristics such as education 
credentials, skills, or work experience. Similarly, behind the average were a few 
people who benefited a lot and many who did not benefit at all, as well as some 
who made modest gains close to the average. Consequently, the research is not 
much help for caseworkers who are attempting to assist individual welfare 
recipients. 

Second, many programs of the 1980s were designed to "let the labor market 
decide" which participants were and were not immediately employable (or job- 



^For a review of the research findings on welfare employment programs in the 1980s, see J. 
Gueron and E. Pauly, Frotn Welfare to Work (New York: Russell Sage Foundation, 1991). 

^See, for example, M. Bane and D. Ell wood. The Dynamics of Dependence: Tlie Routes to Self- 
Sufficiency (Cambridge, Mass.: Urban Systems Research and Engineering, Inc., 1983). 

%ee D. Friedlander, Subgroup Impacts and Performance Indicators for Selected Welfare Employtnent 
Programs (New York: MDRC, 1988). 
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ready) by prescribing a job-search activity first for most. As a result, little research- 
based information is available about other ways to assess employability. 
Furthermore, there have been no "side-by-side" tests (i.e., tests of matched groups 
in one location) comparing programs that start with individualized assessments to 
those that start with job search. Careful evaluations of different assessment strategies 
and instruments in wel fare-to- work programs have not been carried out, either/ 

Which Comes First: The Program or the Participant? 

JOBS administrators also face another dilemma: Should the objectives and 
designs of JOBS programs drive assessment, or should assessment findings about 
JOBS participants drive program designs? These alternatives reflect two very 
different ideas about participant assessment: 

1. Assessment is a method for routing eligible individuals into different service 
components that are designed to accomplish the overall goals of the program—for 
example, reducing welfare expenditures, increasing the earnings or income of 
welfare recipients, or reducing poverty-through their aggregate effects on groups. 

2. Assessment enables program staff to identify the services most likely to enable 
each individual to get a job, get a good job, get off welfare, or get out of poverty. 

In the first case, predetermined program goals and designs dictate assessment 
approaches: People are fitted to the program. In the second, individuals' 
characteristics and circumstances dominate the process: The program is fitted to 
people. 

In JOBS, as in most employment and training programs, this dichotomy is not a 
sharp one. Administrators design programs on the basis of both their clients' needs 
and their program's overall objectives. Their capacity to individualize program 
responses is limited, although program services are often adjusted over time on the 
basis of operational experience. Still, the distinction is important. Assessment 
approaches that are intended to maximize benefits to clients do not necessarily 
produce the same decisions as approaches that are intended to maximize program 
goals. 



^See Gueron and Pauly, From Welfare to Work. An evaluation of different approaches to JOBS, 
being conducted by MDRC under a contract with the U.S. Department of Health and Human 
Services, will compare the effects of up-front job-search models emphasizing labor force attachn\ent 
and up-front assessnrtent models emphasizing human capital development through education. It will 
also measure the effects of assessment-based approaches under different circumstances. 
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A Framework for Thinking 
About Assessment 

Three considerations inform the discussion of JOBS administrators' choices 
throughout the remainder of this paper: (1) the goals of JOBS programs, which can 
vary; (2) the nature of "evidence" about assessment that JOBS administrators might 
want in order to weigh their choices; and (3) the relationship between assessment 
and program operations, which is especially critical to adnrdnistrators because the 
JOBS law ties program funding to certain program performance standards. 

Describing assessment in JOBS, the Family Support Act discusses its purpose 
rather than details of an assessment process: 

The State agency must make an initial assessment of the educational, child care, 
and other supportive services needs as well as the skills, prior work experience, 
and employability of each participant in the program . . . including a review of 
the family circumstances. The agency may also review the needs of any child of 
the participant. On the basis of such assessment, the State agency, in consultation 
with the participant, shall develop an employability plan for the participant. The 
employability plan shall explain the services that will be provided by the State 
agency and the activities in which the participant will take part under the 
program, including child care and other supportive services, shall set forth an 
employment goal for the participant, and shall, to the maximum extent possible 
. . . reflect the respective preferences of such participant. The plan must take into 
account the participant's supportive services needs, available program resources, 
and local employment opportunities.® 

Thus, in one program, "assessment" might refer to a quick screening procedure 
in which participants' recent work experience or high school credentials are 
compared to the stated eligibility criteria for particular program components. In 
another program, assessment could be an intensive and lengthy series of counseling 
sessions in which staff seek to motivate participants, raise their self-esteem, and 
explore their career options before selecting a program activity. 

In the first example, participants might be asked to complete a form that 
describes their school and work histories, their child care and other supportive- 
service needs, and their job goals. The process of turning this information into an 
employability plan and an assignment to a JOBS program component might be as 
simple as asking participants which activity tliey prefer. In the second example, 
several staff members might spend days or weeks with a group of clients gathering 
information about their capabilities and preferences through discussions, testing, and 
observation. Case conferences might follow, leading to recommendations that would 



*P.L. 100«485, Sec. 201(b), creating Sec. 482(b)(1) of tl\e Social Security Act. 
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be translated into formal contracts between participants and the program. 

The basic dimensions of assessment, and how and why these can differ from 
program to program, are discussed in this chapter. 

Ends and Means: The Influence of JOBS 
Goals on Assessment Approaches 

While the intent of the Family Support Act is for states to develop JOBS 
programs that offer a variety of employment-related services, reach a larger 
proportion of welfare recipients than in the past, and target resources to the more 
disadvantaged recipients, the law leaves to states many of the decisions about how 
to put programs together. It outlines the necessary components of a program, 
requires certain plans and procedures, and offers funding incentives that are based 
on standards for participation by eligible individuals and expenditures by states for 
targeted groups. In response, states are building JOBS programs that emphasize 
different goals. These include: 

• Getting more welfare recipients working; 

• Increasing the earnings of welfare recipients; 

• Making welfare families better off financially; 

• Getting welfare recipients jobs that will move them out of poverty; 

• Reducing welfare costs and /or caseloads; 

• Saving the government money; and 

• Getting long-term recipients into jobs and off welfare. 

These objectives lead to different program designs. For example, JOBS programs 
can stress job-search and job-placement services to move people quickly into jobs 
(sometimes called "labor force attachment" program models), or longer-term 
education and training for human capital development. A third alternative is to 
concentrate services of both kinds on certain groups of welfare recipients. Research 
to date suggests that some models get more people working, and thus reduce 
welfare costs, but do not lead to better jobs or improvements for the most 
disadvantaged groups on welfare; other models inaease the income of moderately 
disadvantaged people who might have gone to work anyway but do not lead to 
welfare savings. None of the programs tested so far has succeeded in getting 
significant numbers of welfare recipients out of poverty.^ 

Program design in turn influences assessment. For example, a program aimed at 
getting long-term recipients into jobs and off welfare may emphasize basic skills 
education, work experience, and vocational skills training. Such a program needs an 
assessment system that identifies the long-term welfare group, diagnoses the 
employability problems characteristic of the group, and matches participants to an 
appropriate mix of instruction and support. Alternatively, a program aimed at 
placing more welfare recipients in jobs might require job search as a first step for 
most of those eligible, with an initial assessment geared to supplying supportive 



^See Gueron and Pauly, From Welfare to Work, p. 36. 
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services that will enable them to participate in this activity. In short, assessment 
systems constitute one means for implementing varying JOBS program models, 
which in turn are the principal means for achieving varying JOBS goals. 

This leads to an important lesson for JOBS administrators about untangling ends 
and means: If the overall objectives and service focus of a JOBS program are not 
clearly reflected in its assessment procedures, the participation of eligible 
individuals in specific program components, and the overall flow of clients 
through JOBS activities, may be quite different from the assumptions of program 
planners— possibly so different that an unintended program model emerges from 
assessment practices. For example, when programs with a goal of immediate 
employment for the maximum number of welfare recipients include literacy testing 
in initial assessments, test results often matter most in determining what services are 
provided first. The programs can end up serving more participants than planned in 
education components, resulting in an education-focused model. In contrast, 
programs designed to steer participants with low literacy skills toward education 
can end up with an employment focus if welfare grants are very low, immediate 
jobs are a priority, and client preferences weigh heavily in the assessment process. 

Evidence About Assessment: What Standards 
Should JOBS Administrators Apply? 

JOBS administrators want to know which assessment approach will increase the 
effectiveness of their program given their goals, the characteristics of the groups 
targeted, and local conditions. Some of their other questions about welfare-to-work 
programs can be answered by findings from evaluations that compare groups of 
program participants with groups of individuals who have similar characteristics but 
do not participate.^^ Assessment approaches have not been evaluated by this 
method, however. 

A key reason for this is the difficulty of designing evaluations that can 
distinguish the results of assessment from the results of services to participants after 
assessment. Assessment is generally the first in a chain of events expected to 
culminate in jobs, welfare case closings, or increased income. Consequently, the 
contribution that assessment makes to a program's impact on the employment, 
eamings, welfare status, and other circumstances of participants is difficult to 
measure, even when a study employs a control group.^' 



^^Because welfare recipients may find jobs, leave welfare, and experience other changes in their 
circumstances without participating in employment programs, the groups slated to receive program 
services and the control groups in such evaluations are often formed through a lottery-like random 
assignment process that ensures a clear picture of what happens to people under two 
conditions-when they are eligible for a program and when they are not. See Gucron and Pauly, 
Front Welfare to Work, for a review of such evaluations. 

"An exception to this is when programs place a job-search activity first to '*Iet the labor noarket 
decide" whether participants are employable, as many WIN and WIN Demonstration programs 
operated in the 1980s did. In some of these program models, the servico-job-search assistance— and 
initial employability assessment were considered the same activity. 
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In the absence of evaluation results, what should JOBS administrators know 
about assessment approaches to judge their value? Ideally, the answer to one 
question: Does assessment increase the ability of program staff to predict which 
eligible individuals will and will not benefit from program services? If an 
assessment approach has positive predictive power, then administrators would 
expect to see high rates of participation in and completion of individual program 
components. Assuming that the components themselves are well run and well suited 
to participants, administrators would also expect to see high rates of "success," as 
defined by the program's overall objectives. At the same time, powerful assessment 
techniques would not exclude from specific services, or from a program, individuals 
who could benefit. In short, effective assessment accurately matches clients to 
services in two ways: by not referring to a particular service those who are 
unlikely to benefit from it, and by not failing to refer those who are likely to 
benefit. 

Judging the predictive power of different assessment approaches is a substantial 
challenge. "Situational" issues, such as ill health, family problems, and disruptions 
in child care arrangements, make it difficult for staff to identify the clients who will 
successfully complete a program activity or find a job.'^ In addition, personal 
characteristics like motivation, which are very difficult to measure, may have a 
strong impact on an individual's participation and future employment. 

Predictability is also affected by employers' hiring, supervision, and promotion 
practices. In the chain that leads from assessment to program participation to 
employment, the employer plays an important role. But people who study 
employment and training programs and the labor nrvirket have not been able to 
ascertain just who hires whom with much precision.^^ Also, despite a growing 
body of literature about the basic skills that employers seek, information about the 
skills needed to enter specific occupations, and about "pre-training" requirements for 
occupational training, is hard to come by.'^ 

If JOBS administrators cannot judge the predictive power of the assessment 
approaches, they might consider several fallback standards: 



'^An evaluation of a voluntary group job-search program for welfare recipients that operated in 
Louisville, Kentucky, in the late 1970s provided cautionary evidence about narrow targeting policies 
in such programs. The dennonst ration program studied was intended to involve a diverse group 
of welfare recipients (primarily single mothers with school-age children) and screen out only those 
with the nnost severe barriers to program participation and eventual employment. While no one 
who wanted to participate was excluded on the basis of her job-readiness, the program staff were 
asked to rate the employability of program registrants. The evaluation found little relation between 
staff ratings and outcomes for program-eligible individuals. Those judged not job-ready were 
represented annong program "no shows," "dropouts," "completers," and "successful job-seekers" in 
approximately the same proportions. See J. Gould-Stuart, Welfare Wotnen in a Group Job Seardt 
Program (New York: MDRC, 1982). 

'^See, for example, F. Levy and R. Mumane, "Earnings Levels and Earnings Inequality: A Review 
of Recent Trends and Proposed Explanations," Journal of Economic Literature, forthconrdng. 

'*See R. Yudd and D. Nightingale, The Availability of Information for Defining and Assessing Basic 
Skills Required for Specific Occupations, Urban Institute Policy Menwrandum (Washington, D.C.: 
Urban Institute, May 1990). 
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1. Do fonnal assessment approaches provide program staff and participants with 
information that they understand, value, and use in making choices about 
services? If JOBS program staff say they don't use the information they receive 
from outside assessment agencies or it does not add to their ability to help clients 
choose, its value for them is questionable. Participants in the same program, 
however, might learn more about their abilities, in a more acceptable way, 
through formal assessment activities than through meetings with staff. 

2. Does assessment lead to targeted groups receiving the services intended? 
Assessment methods should serve the ends of the program by identifying 
targeted groups and getting individuals in these groups the services intended for 
them. If assessment is effective by this standard, JOBS administrators would 
expect to see high rates of enrollment in the program for target-group members. 
Further, if program plans specify particular services for subgroups among those 
targeted, members would be accurately sorled into the appropriate service 
components. 

3. Does assessment increase the consistency of program decisions about "who 
goes where"? Administrators are often interested in ensuring that different staff 
members follow the same procedures and make similar decisions under similar 
conditions. If an assessment technique yields consistent results, administrators 
would expect to see that individuals with similar characteristics receive similar 
services in their programs. This helps ensure that service providers get the types 
of clients for whom their programs are presumably most effective. 

4. Does assessment change the nature of decisions that program staff make about 
"who goes where"? In other words, does the introduction of a particular 
assessment approach make a difference-any difference at all— in program 
operations? JOBS administrators should try to answer this question when they 
introduce new assessment approaches that are more complex, more expensive, 
and /or more time-consuming than previous practices. 

Operational Considerations in Choosing 
Assessment Approaches 

Although it may be difficult for JOBS administrators to judge whether and 
how much assessment improves the effectiveness of their programs, at a 
minimum, assessment should not hinder program operations. Assessment is a 
hindrance if it substantially reduces program participation or the funds available for 
program services, or if it creates conflicts between the agency that enrolls 
participants and assigns them to JOBS activities and the agencies that deliver 
services. Operational issues are discussed in detail in Chapter 8, but three points 
illustrate their role in a framework for thinking about assessment: 

1. Lowered participation in JOBS programs. Assessment is the gatekeeper that 
gives or withholds access to other program components. Thus, the time and 
effort involved in assessment will affect how quickly individuals participate 
after they become eligible for JOBS and the proportion who eventually 
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participate. Lengthy assessments that require several meetings between clients 
and staff lead to higher dropout rates than simpler assessments. Some programs 
that offer extensive career exploration activities, or that schedule a battery of tests 
early on, report high dropout rates or frequent interruptions in attendance during 
assessment. Further, initial assessments that involve more than one agency *lose" 
more participants than those requiring only one stop. On the other hand, quick 
assessments do not guarantee higher participation rates. If JOBS program 
enrollees are steered toward activities they are not prepared to perform 
adequately, or they are not interested in, dropping out can still be a problem. 

2. Inter-agency conflicts. In JOBS programs, the welfare agency is responsible for 
setting and implementing policy on "who goes where." But most JOBS 
components are operated by other organizations, often for nonwelfare 
populations as well. These service providers have entry criteria that are 
intended to admit participants who can succeed by their standards, based on 
their experience, and to screen out participants who are likely to fail. When the 
JOBS assessment process does not take these entry aiteria into account the 
result can be inter-agency conflicts over which participants should be served 
in which components. Participants who are referred and then rejected may be 
lost to the program. 

3. Resource constraints. Many things determine the cost of assessment to JOBS 
programs, including the number of people assessed, how much and what type 
of testing is done, and how much time staff take to do the assessment and 
develop an employability plan. Testing forms, scoring sheets, and computer 
devices available commercially are additional program expenses if JOBS 
administrators choose a test-based approach, as are the staff training, facilities, 
and equipment needed to use them. If JOBS administrators have limited funds 
for assessment, they need to consider approaches from a fiscal perspective. 

Dimensions of Assessment: How Approaches Vary 
Vfhat Is Assessed: Broad Areas and Specifics 

Although the term "assessment" is often used as though it referred to a single 
data-gathering and dedsion-n\aking process, an employability assessment can 
encompass many topics. These may be addressed in more or less depth, in different 
sequences, and at different points in time. Among the broad areas on which an 
assessment might focus are participants' (1) potential for immediate employment, 
or job-readiness; (2) basic literacy skills;^^ (3) vocational skills, aptitudes, and 
interests; and (4) supportive-service needs, including, if JOBS officials choose, a 
review of the needs of participants' children, which could affect their employability. 

Within tliese broad categories, what administrators deem relevant to an 
employability assessment will vary widely. In JOBS programs that emphasize 



^^he temis "basic literacy skills/' "literacy skills," and "basic skills" are used interchangeably in 
this paper. 
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immediate employment and are operated in areas with low unemployment rates and 
numerous employers, initial assessments may be limited to a review of participants' 
health and child care needs. In contrast, programs focused on getting participants 
into jobs that will provide stability, fringe benefits, and a good income, or those 
operated in more difficult economic environments, may include a review of 
participants' work histories and their reasons for leaving previous jobs. 

Assessments of basic literacy skills can include math, reading, or both, and these 
can be broken down into computation, problem-solving, comprehension of prose, 
interpretation of documents, and English-language fluency. Similarly, explorations 
of participants' vocational skills, aptitudes, and interests can cover a wide range of 
occupations or be limited to occupations in which training is available locally, jobs 
are projected to increase, or entry wages are high enough to support a family. 

Although this paper focuses on assessing the characteristics of JOBS participants, 
other factors will also influence the impact of programs, such as the demands and 
expectations of local employers, job availability, the entry requirements of training 
vendors, and the availability of child care and other supportive services in the 
community. 

Whe7t Assessment Is Conducted: Sequencing and Staging 

The order in which an assessment addresses employability topics depends on 
priorities. For example, JOBS programs that place priority on getting welfare 
recipients into the labor force may first explore participants' work experience, their 
desire to work, and opportunities in the local job market. Programs that emphasize 
education as the route to self-sufficiency may start with a test of basic literacy skills, 
steering those with low scores to literacy programs and delaying assessment of long- 
term vocational goals. Other progranns may delve deeply into all aspects of 
participants' employability before formulating an initial plan. In such an assess- 
ment, school experiences, literacy skills, work experience, vocational goals, 
vocational aptitudes, knowledge of the world of work, attitudes toward work, and 
job-search skills may receive equal attention. 

JOBS administrators can also choose different assessment sequences over the 
course of a multi-stage program. Although the wording of the JOBS law implies that 
a range of topics must be covered at least broadly in an initial assessment, in-depth 
assessment strategies can be introduced later. Obviously, when some topics are 
explored in more depth at later program stages, participants who leave welfare after 
an initial activity will not receive as extensive an assessment as those who stay 
longer. 

How Information Is Gathered 

Many different data-collection instruments and methods can be used to evaluate 
participants' skills and needs in each of the four areas described above. 
Standardized tests are typically used to assess literacy skills and vocational aptitude, 
but there are also other methods of collecting this information, such as interviewing 
participants, asking them to demonstrate their literacy by filling out forms, and 
reviewing their school records. In some progranns, staff members observe 
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participants in workshops or simulated work settings to determine their job- 
readiness. 

The extent to which participants' preferences are explored depends partly on the 
instrument chosen. As noted above, staff may simply ask participants about the 
occupations, training courses, or program activities tliey prefer. Or they may offer 
formal or informal education and counseling— limited or extensive-before making 
an assignment. 

Who Is Assessed 

Assessment procedures may vary by participant subgroup if program services are 
intended to vary. In a JOBS program that provides only work experience or 
education for the principal wage-earners in two-parent AFDC households (usually 
men), the assessment process needed to choose between these activities may be 
different from (and simpler than) one designed for single mothers. Similarly, 
because the JOBS legislation specifies education as the primary service for teenage 
AFDC parents who have not finished high school, assessment for this population 
may have a different emphasis and sequence than assessment for adults. 

Assessment Decisions: Making Assig7tments to JOBS Activities 

The criteria that a JOBS staff member uses to assign a person to an activity or 
component after assessment may be formal, as when objective participant 
characteristics point to a particular service component. Or they may be general 
guidelines, allowing staff and participants flexibility to choose services on a case-by- 
case basis. Formal criteria can set minimum entry standards, as by requiring 
participants to have at least a seventh-grade reading level to enter vocational 
training, or they can focus on participant characteristics, as by identifying all those 
with reading skills below the ninth-grade level, who might then be assigned to an 
education activity. A combination of the two strategies is also possible: using 
minimum entry criteria for some JOBS components and mandating participation in 
designated activities for individuals with certain characteristics. Assignment criteria 
can also be devised to match participants with certain demonstrated competencies 
or behaviors to JOBS activities. For example, regular attendance in a short pre- 
employment or job-readiness course may be made a prerequisite for entry into a 
long-term training program. 

Using any of these approaches entails several levels of decision-making. First, 
program staff may decide (with participants, in some cases) on the general type of 
activity that seems appropriate, such as education, vocational training, job clubs, on- 
the-job training (OJT), or work experience. Next, if there are options, a specific 
course, vendor, or placement may be selected. At yet another level, staff may need 
to decide about the appropriate hourly or weekly schedules for participation and the 
fit between a participant's program assignment and her supportive-service needs. 

Costs of Assessment 

As more topics are covered in great depth, more extensive information-gathering 
methods are prescribed, and more time-consuming procedures are employed for 
matching participants to services, JOBS assessment becomes more expensive. The 
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range of possible costs is very wide: from less than $50 per participant to hundreds 
of dollars. 

Theoretically, investments in assessment can improve the cost-effectiveness of 
JOBS programs by targeting employment-related services to the participants who 
will benefit most from them. There may be a point, however, beyond which 
additional funds expended do not improve the fit between participants and 
programs. Because assessment is sometimes in direct competition for JOBS funds 
with case management and employment-related activities, program administrators 
designing assessment systems must consider the possibility of diminishing returns. 

There is limited research evidence to guide JOBS administrators on trade-offs 
between investing in assessment and investing in other JOBS activities, but strategies 
vary according to program goals: 

• Programs that focus on immediate employment, provide up-front job search, and 
are designed to reach a broad segment of the eligible population typically make 
minimum investments in assessment at first, reserving in-depth assessments for 
participants who fail to find a job. 

• Programs that provide relatively expensive services for a selected segment of the 
eligible population typically invest more in assessment, seeking to ensure that 
participants have the prerequisite skills to complete training and are motivated 
to do so. 

• Programs that offer a mix of services typically screen participants first for basic 
literacy skills. Additional assessment is minimal for some participants and 
considerable for others, depending on their characteristics or on the types of 
services being considered. 

Staging assessment procedures is one way to achieve a mix of assessment 
expenditures for different participants. Another is to match the intensity (or cost) of 
assessment to the intensity (or cost) of employment-related services: High-cost 
assessment strategies might be reserved for making decisions about assigning 
participants to high-cost services (such as education or training), and less costly 
strategies might be used for assigning individuals to low-cost services (such as job 
search or work exp>erience). 

Local Variation 

From the state perspective, another dimension of JOBS assessment is local 
variation: State JOBS administrators must decide whether all local JOBS programs 
should use similar assessment instruments and decision-making methods for activity 
assignments. While standardization may benefit program planning, there are also 
reasons to allow local variation. For example, if JOBS program designs differ from 
locality to locality, perhaps assessment practices that fit program designs should 
vary as well. Local assessment practices and instruments might be expected to differ 
even when the basic program components and the flow of participants through a 
sequence of JOBS activities are similai^for example, when JOBS programs use 
service providers with varying entry requirements. Differences in labor markets are 
another reason to adapt assessment practices to local conditions. 
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Summary: An Assessment Framework 

Each dimension of assessment described above encompasses a set of choices for 
JOBS administrators. Based on the selections they make, administrators can design 
assessment systems that are more or less intensive, lengthy, comprehensive, and 
uniform. Table 1 summarizes their choices in four major assessment areas (listed in 
the first column): job-readiness (immediate employability), basic literacy skills, 
vocational aptitudes and interests, and supportive-service needs. In the other 
columns, each entry next to a bullet (•) represents a possible feature of an 
assessment system. JOBS administrators can "irux and match" these features, 
structuring an assessment in the way that makes most sense for their prograins. 

While many combinations of assessment features are theoretically possible, they 
tend to occur in constellations associated v^ith a fev^ basic JOBS program designs. 
Assessments designed to evaluate all areas of all participants' employability upon 
program entry, using standardized tests and other staff-intensive data-collection 
strategies, are found in a type of program that is sometimes called (not surprisingly) 
an up-front assessment model. In another basic program type, called up-front 
education, initial assessments focus on basic literacy skills. In these tvy^o models, 
participants' potential for immediate employment is a second-order question, usually 
explored if they have completed high school or a GED (General Educational 
Development, or high school equivalency) program, passed a literacy test, or do not 
v\^ant more schooling. Another approach, called up-front job search, puts an 
assessment of immediate employability, or job-readiness, first. Figure 1 displays 
these three models. 

Programs that offer a fixed sequence or a very limited choice of initial 
activities— as in the up-front education or up-front job-search models— are likely to: 

• Make initial assignments on the basis of v^hether participants' objective 
characteristics match specific criteria; 

• Involve participants relatively little in the decision-making process; and 

• Stage assessments to give more attention and choice to participants who have 
completed an initial program component and are eligible for a greater range of 
activities. 

Programs that offer participants a wide range of activities when they enter JOBS-as 
in the up-front assessment model— are likely to: 

• Include tests for vocational aptitudes and basic literacy skills in an initial 
assessment; and 

• Give both staff and participants more discretion in choosing program activities. 
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FIGURE 1 

THREE PROGRAM MODELS 
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Chapter 3 



Assessing Job-Readiness 

Avoiding long-term welfare dependency is the goal of the Family Support Act 
reforms, and "work, child support, and fanuly benefits" are cited in the acf s 
statement of purpose as the means to achieve this. Lavmiakers and JOBS program 
adniinistrators have different visions of how much work, at what wages, with what 
initial investments in education and training can and should be expected of welfare 
recipients, however. JOBS programs across the country reflect these differences. 
Early assessments of participants' job-readiness, or potential for immediate 
employment, are central to two types of JOBS programs: (1) those that give high 
priority to increasing the number of welfare recipients who go to work, and (2) 
those that operate within severe resource constraints. Some participants require less 
costly program assistance to find work than others; thus, the ability to distinguish 
those with the potential for immediate employment from those who require longer- 
term services in order to get a job theoretically enables programs to target their 
resources efficiently. When the allocation of scarce program resources is an 
important concern, JOBS administrators may seek to use the assessment process to 
identify participants who need little assistance in finding employment, in order to 
reserve resources for those who need more. 

JOBS administrators face a challenge in formulating a distinction between these 
two groups that can be translated into effective program responses. While job- 
readiness can be defined to be consistent with a range of program goals, deciding 
on an appropriate program service for a job-ready group is not easy. The usual 
services provided to "job-ready" participants in welfare-to-work programs of the 
past— job-search assistance or job-placement activities-do not necessarily benefit 
job-ready program participants. 

Defining Job-Readiness 

Wage goals for JOBS parhcipants are a nnajor determinant of who is judged job- 
ready. In one program, job-readiness may be defined as the ability to obtain a job 
that produces no net loss of cash income,^^ whereas in another it may mean the 
ability to obtain a job that makes a family substantially better off than it is on 
welfare. In some programs, a participant is considered job-ready if she can work at 
any job, including one that reduces her AFDC grant but does not result in closing 
her welfare case. Of course, the variation across states in AFDC grant amounts and 
wages for entry-level work has a sizable effect on the welfare-versus-work equation. 
In a low-grant state, minimum-wage jobs can increase net income, whereas in a 
high-grant state, welfare recipients may have to earn almost twice the minimum 
wage to be disqualified from AFDC. 



'^JOBS program regulations state that participants who would experience a net loss of cash 
income have "good cause" for 'refusing to accept a job. See 45 CFR 250.35(c). 
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Assessments of immediate employability can also be designed to give the greatest 
weight to participants' preferences. If a participant wants to work, she may be 
judged job-ready regardless of her education, previous work experience, or other 
characteristics. 

Definitions of job-readiness also hinge on how much assistance is available for 
those in need of the fewest services. Job-placement or job-search components can be 
more or less structured and involve more or less instruction and support. In some 
cases, job-ready individuals are referred to a U.S. Employment Service agency or 
directed to conduct an independent job search without any initial JOBS service. 
Program adnunistrators may also prefer imjnediate employment for some 
individuals as a method of developing skills and work habits. For instance, a 
participant may combine part-time employment with welfare receipt and part-time 
education or training. In some programs, staff may not be asked to judge job- 
readiness at all: If JOBS administrators believe that tlie workings of the labor market 
are the best indicator of job-readiness, they may rely on up-front job search to reveal 
which participants can find work. 

A Job-Readiness Paradox: Who Benefits from Job Search? 

JOBS programs that strive for accuracy in assessing job-readiness by one 
standard— correctly predicting who can go to work with minimal assistance— may 
undermine their achievements by another standard-making a difference through 
program services for JOBS participants. This paradox stems from two facts. First, 
as noted above, job-search assistance-frequently intensive, although lasting only a 
few weeks— has been the usual service provided by welfare employment programs 
to participants considered job-ready. Second, programs featuring job-search 
assistance have not made a difference in the employment, earnings, or welfare 
receipt of the most job-ready participants, typically, first-time AFDC applicants with 
recent work experience. A more disadvantaged group of welfare recipients benefited 
from these programs, and the experiences of control group members in evaluations 
indicate that the job-ready participants would have gone to work even without help 
from the programs studied .^^ 

It is difficult to know, without comparison studies, whether the job-search 
assistance offered in programs of the 1980s was too much, too little, or not the right 
kind^^service for people with recent work experience.^® However, the research 
findings suggest that predicting whether someone has the potential for immediate 
employment is not the same as predicting whether someone will go to work as a 
result of job-search assistance. 

This distinction leads to a choice for JOBS administrators between defining job- 
readiness on the basis of the potential to get a job or the potential to benefit from 



^'See Gueron and Pauly, From Welfare to Work. 

^*Studies of individual and group job-search programs operated under different conditions in 
Louisville, Kentucky, from late 1973 through mid-1981 suggest that the effects of the two 
approaches are similar. See C. Wolfhagen and B. Goldman, Job Seardt Strategies: lessons from tJie 
Louisville WIN Laboratory (New York: MDRC, 1983). 
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job-search assistance. If they define job-readiness narrowly-for example, as having 
the credentials, experience, and personal goals associated with high rates of job- 
finding-a critical program design question follows: whether to include or exclude 
this group from job-search services, or even to limit such services to participants 
who nrieet this definition. An alternative is to define job-readiness broadly and 
include some people with less apparent employability potential who might be able 
to find work with the help of job-search services. 

Assessing Job-Readiness When Indicators 
Are Not Qear-Cut 

The factors that might be considered in a definition of job-readiness include: 

• Participants' previous work experience, including: 

Wages, stability of employment, reasons for leaving, type of work, and job 
performance 

Availability of similar jobs in the local labor market 

Stability of similar jobs and opportunities for upward mobility 

• Participants' preferences, including: 
Type of job or occupational choice 
Working conditions and schedule 

Education or training as an alternative to work or in combination with work 

• Participants' education levels and basic literacy skills, including: 
High school diploma or GED certificate 

Highest grade completed 
English-language proficiency 

Minimum proficiency for types of jobs available or desired 
Previous job training 

• Participants' income needs, including: 

Earnings needed to ^T^reak even" compared to existing benefits 
Earnings combined with AFDC needed to exceed the AFDC grant 
Amount that would provide a substantial net increase over the total welfare- 
related benefit package 

• The program's job-finding services, including: 
Individual job search with no instruction or support 
Job clubs with instruction and support 

Job placement (e.g., referrals to job openings) 

It is relatively easy to make decisions about participants' potential for immediate 
employment when all the job-readiness factors considered point in the same 
direction. Usually they do not. Participants whose potential for immediate 
employment is uncertain— and who might fail to get a job as a result of an 
employment search, or who jnight fail to keep a job and return to welfare, or who 
might only be able to get a job that provides the same income as welfare-are the 
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ones for whom questions of program goals and philosophy most influence 
assessment decisions. For example: 

• Is it a policy to reserve scarce program resources for participants who are most 
likely to stay on welfare a long Hme? If so, then broader definiHons of job- 
readiness, leading to more participants being judged job-ready and referred to 
job search or job placement, might be adopted. 

• Does the program have enough alternatives to job search or job placement to 
accommodate all those judged not immediately employable? If not, a broader 
definition of job-readiness might be indicated. 

• Is reducing welfare expenditures a higher or lower priority than increasing the 
earnings or net income of welfare recipients? Because these two goals suggest 
different program models, job-readiness definitions and decisions may be geared 
to different services. Typically, programs aimed at reducing welfare costs assign 
individuals with a wide range of characteristics to job-search /job-placement 
activities, whereas programs aimed at increasing participants' income assign 
more individuals initially to education, job training, or other "human capital 
development" activities. 

• Is short-term job-holding considered a step into the world of work, or a "failure" 
experience? Employment and training professionals disagree about the value of 
entry-level jobs tliat pay low wages, do not offer fringe benefits, lack career 
opportunities, or are seasonal. In programs that give priorit)' to holding any job, 
definitions of job-readiness are usually broad and the risk of failure is 
downplayed. 

• How much weight is given to participants' preferences? If the answer to this 
question is "a great deal," program staff who are uncertain about a participant's 
job-readiness may rely on her interest in trying job search/job placement or 
another job-preparation activity. 
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Research suggests that there is a trade-off between maximizing immediate 
job placements and maximizing program impacts (the difference that a 
program makes for participants). Therefore, program administrators should 
define job-readiness to fit their program goals. For example, if JOBS programs 
adopt a relatively narrow definition of job-readiness, the group of 
individuals so designated will be small but the rate of job-finding among 
them will likely be high; however, a narrow definition of job-readiness will 
exclude some more disadvantaged individuals who could also go to work 
with minimal assistance. Conversely, a broad definition of job-readiness 
will create a larger group to be given assistance, but the rate of job-finding 
will be lower. 

Definitions of job-readiness will also have a major impact on program flow 
and resource allocation. The more broadly the term is defined, the greater the 
proportion of the caseload that will usually be assigned first to job-search or 
job-placement activities, which are typically among the least expensive 
program services on a per participant basis. If employability is narrowly 
defined, a larger proportion of participants will usually be assigned to 
activities such as education and training that entail a greater investment of 
time and resources per participant. 

In choosing assessment approaches, JOBS administrators should consider 
whether their program objectives point to a broader or narrower definition of 
job-readiness, for example: 

Important Program Goal Definition of Job-Readiness 

Maximizing the number of participants 

who find immediate employment Broad 

Maximizing welfare savings Broad 

Maximizing the rate of job entry for 

participants in a job-search activity Narrow 

Making substantial irivestments in human 

capital development Narrow 

Reserving expensive services for those 

who cannot go to work without them Broad 

Providing education and training services to 

all participants who could benefit from them Narrow 

Maximizing client and caseworker choice Broad 
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Assessing Basic Literacy Skills 

Concerns about the increased likelihood of long-term welfare dependency among 
young mothers who are high school dropouts, as well as about a mismatch between 
the skills that workers entering the labor market have and those that employers 
need, have prompted many JOBS administrators to emphasize basic skills education 
in their programs. The objective of these programs is first to improve basic skills 
among welfare recipients and then to turn education gains into increased earnings 
and self-sufficiency. 

Provisions of the Family Support Act mandating education as a JOBS activity for 
some segments of the welfare population give a central role to literacy assessment. 
The JOBS legislation specifies that if individuals age 20 or over are required to 
participate, and they do not have a high school diploma or equivalency degree, their 
employability plans must include an education activity, unless these participants 
have attained a basic literacy level or their long-term employment goals do not 
require a high school degree.^^ Basic literacy is defined in the program's implemen- 
tation regulations as functioning at least at an 8.9-grade level.^° The law also 
stipulates that, to the extent permitted by resources, states must require AFDC 
custodial parents under age 20 who do not have a high school diploma or GED to 
participate in an appropriate education activity, regardless of the age of their 
children.^^ 

Standardized tests designed to measure reading and math skills are widely avail- 
able and are often scored in relation to school grade levels, thus simplifying this 
assessment task in JOBS. However, the relationship between basic skills and job per- 
formance is poorly understood, and there is considerable uncertainty about the most 
appropriate ways to measure the kinds of basic skills that are required on the job. 
As a result, it is relatively easy to measure JOBS participants' basic literacy, but 
difficult to decide on an effective program response to grade-equivalent literacy 
results. 

JOBS administrators making decisions in this area must first confront the 
dilemma discussed above: Should the objectives and designs of JOBS programs 
drive assessment, or should assessment findings drive program designs? When basic 
skills testing in JOBS is a universal and early step, the results tend to push programs 
toward large-scale remediation efforts. This is because, while most women on wel- 
fare have completed high school or received a GED, most score well below national 
averages on basic skills tests.^^ Large-scale remediation efforts may not serve the 
goals of JOBS in some states and localities. Administrators need to examine the 



Sec. 482(d)(2) of the Social Security Act. 
^^ISee 45 CFR 250.1. In this context, grade 8.9 means the ninth month of the eighth grade. 
2'See Sec. 402(a)(19) of the Social Security Act. 

N. Zill et al, Welfare Motliers as Potential Employees: A Statistical Profile Based on National 
Survey Data (Washington, D.C.: Child Trends, Inc., Febniary 25, 1991). 
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capacity of local education providers to help welfare recipients improve their skills, 
complete education programs, and go to work before addressing specific literacy 
assessment questions such as: Are standardized tests necessary? Should tests be 
given to all or only some participants? Should literacy testing be a first step in a 
JOBS program? 

Using Standardized Tests 

Information about participants' education history— the highest grade completed 
and whether they received a high school diploma or passed the GED tesMs the 
foundation for a literacy or basic skills assessment. High school graduation or GED 
attainment is a prerequisite for many jobs, and studies show that individuals with 
these credentials earn more than individuals who lack tliem?^ However, high 
school graduates may still be deficient in tlie reading and math competencies 
required on the job, especially if they have been out of school and out of the labor 
force for some time. Among people whose principal language is not English, basic 
skills in English may also be inadequate for employment regardless of education 
level. Therefore, JOBS programs need methods of assessing basic skills beyond 
ascertaining the number of years participants spent in school, certainly for the 
segment of the welfare population without high school diplomas and perhaps for 
high school graduates as well. Increasingly, standardized tests are that method. 

Several types of standardized tests are widely used for measuring the basic 
literacy skills of welfare recipients. One type is academically oriented, measuring tlie 
reading, math, and other skills that are traditionally taught and evaluated in 
elementary and secondary schools, and producing test scores that can be interpreted 
(with caution) as reflecting school grade-level equivalents.^* Tliese tests focus on 
general academic knowledge such as spelling, identifying the main idea in a reading 
passage, grammar, computation, and arithmetic word problems. Most of these tests 
are described as "norm-referenced," because they measure an individual's skills in 
relation to those of a specified group of representative individuals (e.g., a 



^^Sce G. Berlin and A. Sum, Toward a More Perfect Union: Basic Skills, Poor Families, and Our 
Economic Future, Occasional Paper No. 3, Ford Foundation Project on Social Welfare and the 
American Future (New York: The Ford Foundation, 1988). For a comparison of the economic 
consequences of GED certification and high school graduation, see S. Cameron and J. Heckman, 
'The Nonequivaience of High School Equivalents," Working Paper No. 3804 (Cambridge, Mass.: 
National Bureau of Economic Research, 1991). 

^^Many experts regard grade-level equivalents as a poor and inappropriate measure of 
achievement among adults, because most adults— whatever their reading and math achievement 
levels-know a great deal that elementary and secondary students do not. In addition, the basic 
skills requirements of most jobs do not correspond to the achievement levels of elementary and 
secondary students, so grade-level equivalents are not well suited to assessing job-readiness. 
However, grade equivalents have attained wide currency in popular discussions of literacy. Like 
any scale, grade-equivalent results make it possible to distinguish higher and lower achievement 
levels and to recognize extremes. See R. Linn, ^'Quantitative Methods in Research on Teaching," in 
M. Wittrock, cd., Handbook of Research on Teaching, 3d ed. (New York: Macmillan, 1986), pp. 98-99; 
and I. Kirsch and A. Jungeblut, Literacy: Profiles of America's Young Adults, Final Report (Princeton, 
N.J.: National Assessment of Educational Progress, 1986), pp. V-10 and 11. 



Assessing Basic Literacy Skills 



29 



nationwide sample of adults or a nationwide sample of youths in school). Grade- 
equivalent scores are an example of norm-referenced scoring. 

Tests of this type are the TABE (Tests of Adult Basic Education), ABLE (Adult 
Basic Learning Exam), and WRAT (Wide Range Achievement Test). All use a 
multiple-choice format. The ABLE incorporates some life skills questions, but the 
emphasis is on academic skills.^^ 

A second type of test, developed more recently, is designed to measure basic 
literacy skills needed in adult life and in the workplace, rather than in a traditional 
school setting. Although their developers say these tests measure ^'workplace 
literacy" or "functional competencies," it is not known whether the test items 
represent a valid sampling of the literacy tasks encountered by adults. Some are 
norm-referenced; others are described as "criterion-referenced," because they 
measure an individual's performance against a set standard (a "criterion") rather 
than against the scores of previous test-takers. 

Two widely cited examples of such tests are the California Adult Student 
Assessment System (CASAS) and the Tests of Applied Literacy Skills (TALS) 
developed by the Educational Testing Service. CASAS tests, prepared with the help 
of a consortium of education agencies in California, are used in adult education, 
welfare, and other employment and training programs in 25 states. (Outside 
California, they are not always called CASAS tests.) The TALS was designed to 
provide information about adults similar to that provided by the National 
Assessment of Educational Progress of American Youth, which was conducted in 
1985, and the National Adult Literacy Survey planned for 1992. 

CASAS tests are varied and can be adapted for specific program applications. 
One version, the GAIN Appraisal (used in California), is a 50-minute multiple-choice 
test that measures reading and math skills. CASAS also includes listening 
comprehension tests for those with poor English skills and a series of achievement 
tests intended to measure progress in adult education programs. 

The TALS differs from CASAS tests in that it asks questions that require a brief 
written response (often "fill in the blank") rather than using a multiple-choice 
format. In addition, the battery includes tests that assess "prose literacy" (the ability 
to read and interpret information in newspapers, nr^gazines, and books); "document 
literacy" (the ability to identify and use information in application forms, tables, 
charts, indexes, and the like); and "quantitative literacy" (the ability to perform 
numerical operations on information contained in printed materials). 

Comparisons between the academic tests and functional tests are difficult. 
Academic tests have been used longer and more widely to measure the literacy of 
adults. Anecdotal reports suggest that functional competency tests are more likely 
to be accepted by adults, but none of the tests cited here has been validated as a 
predictor of job performance or employability.^^ 



^^See G. Jackson, Measures for Adult Literacy Programs (Washington, D.C: ERIC Clearinghouse on 
Tests, Measurement, and Evaluation, 1990). This document provides concise descriptions of a large 
number of tests that may be considered for use in JOBS programs, including their administration 
requirements. 

^*See Jackson, Measures far Adult Literacy Programs. 




Guidelines for Selecting Standardized Literacy Tests 

Standardized literacy tests come in many formats and lengths and have 
varying purposes and administration requirements. For JOBS program 
administrators who decide to use a test, the options are narrowed by three 
main considerations. 

1. It is practical to choose a test that is relatively short, is easy to 
administer and score, and meets recognized standards of technical sound- 
ness. In general the TABE, ABLE, WRAT, CASAS, and TALS meet these 
criteria. A short test is preferable to a long one if the assessment is intended 
to determine only general achievement level. For more precise diagnostic 
purposes, a longer, more detailed test is required. A number of pro-ams use 
the TABE locator" test, which takes about 20 minutes to administer, or only 
the reading and math portions of the TABE (instead of all seven subtests, 
which take a total of 4Vi hours to administer). California's Greater Avenues for 
Independence (GAIN) Program uses tlie 50-nunute CASAS appraisal test, 
which measures reading and math skills, as an initial literacy screen, rather 
than the longer CASAS achievement tests. Tlie lengtliier versions of the TABE 
and CASAS might be used to pinpoint specific weaknesses and strengths after 
a participant has entered an education program. 

2. Literacy tests should be well matched to the general purpose of the 
JOBS program and the assessment process. For example, because academi- 
cally oriented tests contain items that resemble those in the entry tests used 
by providers of education and job-training services, and to a lesser degree 
resemble items on the GED test, they may be the best tool for assessing 
literacy in programs that refer participants to providers that require specific 
test scores for entry or emphasize GED attainment. It may also be useful to 
administer tlic GED practice test to determine which participants are good 
candidates for GED instruction. However, such tests may not be useful for 
programs that focus on immediate employment. For example, the TABE's use 
of items that require the correct use of the apostrophe points to a level of 
grammatical sophistication that is not relevant to a wide range of jobs. 

3. Selecting tests that will be accepted by education and training 
providers in JOBS will help smooth operations. Multi-agency employment 
and training programs in some locations are plagued by a "retesting" 
phenomenon, which can be wastcful-both of resources and the enthusiasm of 
participants. JOBS administrators should find out how local education and 
training providers assess basic literacy. Tliey may be able to develop inter- 
agency agreements on testing and referral criteria to minimize the retesting 
problem. Such agreements might cover which agency will test JOBS 
participants, which instruments will be used, and the test scores that are 
acceptable for entry into specific program activities. 
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Alternative Literacy Assessment Strategies 

States have considerable flexibility in deciding whether to use a standardized test 
or sonve other method to decide which participants need education and what type. 
The JOBS legislation and regulations do not require testing as an assessnr^ent 
strategy, although the U.S. Department of Health and Human Services' preamble to 
the regulations recommends that, "where appropriate," states use "nationally 
recognized, standardized, or industry-developed tests to determine literacy levels 
and aptitude skills before assigning participants to specific educational and 
vocational training programs."^'' While some states administer standardized 
achievement tests to all eligible program participants, others test only some 
participants, and still others rely primarily on the highest school grade that 
participants completed to judge literacy. In lieu of testing, some programs have 
participants fill out a work history form or other questionnaire as part of the JOBS 
intake/orientation process. According to staff who use this technique, it provides 
an excellent indication of how well participants read, write, spell, use grammar, 
answer questions, and follow instructions. 

Such informal demonstrations of literacy skills can be supplemented by 
interviews in which staff ask questions about the type of reading participants do at 
home or on a job. Similar informal approaches can be devised to judge participants' 
fluency in English— for example, on the basis of their work history, their employers' 
references, and their ability to converse in English during interviews, read and 
answer questions about program-related materials, or fill out forms. 

Advantages and Disadvantages of Standardized 
Tests and Alternative Strategies 

Most standardized tests are easy to administer. They can be given in a group 
setting, are not very costly, and can be taken and scored quickly. In addition, the 
tests cited above (with the exception of the TALS, which has not yet been widely 
used) meet recognized standards of technical soundness and are accepted or used 
by many employment and training, adult education, and welfare employment 
programs.^® 

However, experts uniformly warn against using a single test as the sole basis for 
screening participants for services. They emphasize the importance of other factors 
(such as the individual's employment goal, education background, and motivation) 
in making such decisions. One analyst of basic skills testing for employability has 
warned: 

In studies of the predictive validity of the most widely used basic skills test, the 
Armed Forces Qualification Test (AFQT), it was found that of those that military 
selection policies had predicted to fail in job training and on the job, eight of the 
ten actually performed satisfactorily. Tliese data, for an organization that has 



2^See Federal Register, vol. 54, no. 197, October 13, 1989, p. 42178. 
^*See Jackson, Measures for Adult Literacy Programs. 
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studied this type of assessment for seventy years at a cost of at least $100 million, 
should caution the "gatekeeping" use of basic skills tests in workfare/welfare 
programs, workplace literacy, and JTPA programs?^ 

While tests measuring functional literacy should be better predictors of employ- 
ability-related basic skills than those that measure academic skills, no research has 
demonstrated whether this is true or false.^° 

Many assessment experts and program operators also warn about the "test 
anxiety" that formal testing creates in some people. They believe that requiring a test 
or a battery of tests at the beginning of a program can have a negative effect on 
participants who associate testing with past failures and unpleasant experiences in 
school. Testing may dampen enthusiasm for the program and affect participation 
rates. (One assessment specialist argues, however, that such concerns are 
exaggerated, and that not only testing itself but early testing in an unfamiliar, 
impersonal environment accounts for the problem.^') To reduce test anxiety, 
experts recommend several strategies: delay testing until after participants have had 
a chance to accustom themselves to the program environment; assure participants 
that tests are intended not to screen people out of services but to identify the best 
way for the program to help them; use a short test rather than a long one; and 
ensure that staff members who administer tests establish rapport with the clients 
before giving the test instructions. One program even gives clients "stress 
management" training before administering a standardized test of basic skills. 

Informal assessment strategies have their own advantages and disadvantages. 
They may reduce or eliminate test anxiety, but their results are imprecise and more 
subjective than standardized test scores, partly because there is no clear reference 
group against which performance can be evaluated. 

Informal strategies may work best when a literacy assessment is intended to 
identify people who cannot read or write English, but they cannot pinpoint the 8.9- 
grade level specified in the JOBS regulations as the standard for basic literacy, nor 
are they able to identify people who are likely to pass the GED test with a modest 
amount of instruction. JOBS administrators should also take into account whether 
the education and training providers they work with consider informal assessments 
reliable. In many programs, these providers want evidence of literacy skills con- 
sistent with their entry standards. However, if JOBS participants who are referred 
to education and training providers would be retested anyway, an informal assess- 
ment might be a useful screening strategy for the JOBS program and more cost- 
efficient than administering a formal test. 

The answers to the three key questions identified above- Are standardized tests 
necessary? Should tests be given to all or only some participants? Should literacy 
testing be a first step in a JOBS program?-depend on the specific program model 



Sticht, Testiiig and Assessment in Adult Basic Education and English as a Second Language 
Programs (San Diego: Applied Behavioral and Cognitive Sciences, Inc., January 1990), p. 24. 
^ticht, Testing and Assessment, p. 25. 

^'Telephone interview with Lori Strumpf, Director, Center for Remediation Design, Washington, 
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adopted and on the entry requirements for program components. Universal, up- 
front, standardized testing of basic literacy skills in JOBS might make sense for 
programs that emphasize education and job training rather than immediate 
employment, and that rely on providers of these services that set entry standards 
based on test scores. But the cautions against this practice noted above— e.g,, test 
scores tend to be given more weight in decisions about services than is justified by 
their predictive capabilities— still apply. This method of assessment may not make 
sense for programs that emphasize immediate employment or that rely on 
providers of education and training services that administer their own tests before 
admitting anyone to their programs. 

Education Program Responses to Test Scores 

Many adult education programs use basic skills test scores to identify the type 
of education program an individual should enter. For example, individuals who 
read above the eighth-grade level are often assigned to courses that prepare them 
for a liigh school equivalency degree (GED classes); those reading between tlie 
fourth- and eighth-grade levels are often assigned to adult basic education (ABE) 
classes, which are geared toward improving basic reading and math skills; those 
who test below the fourth-grade level are assigned to basic literacy classes, which 
focus on reading and vocabulary building. Individuals with a very limited command 
of English are referred to English as a second language (ESL) classes. In some states, 
the law bars large vocational-technical education centers from accepting people 
whose test scores fall below specified levels (usually eighth-grade). 

While these are common standards for making decisions about what kind of 
education program is appropriate for people with poor basic skills, JOBS 
adnunistrators also face fundamental questions about how much education to 
provide for whom and whether participants with poor skills should be excluded 
from other activities, including job search. Data on completion rates for GED and 
ABE programs are sketchy, but JOBS staff across the country report that high 
proportions of participants drop out of these programs or stay enrolled for long 
periods without making progress.^^ 

Recent amendments to the Adult Education Act, the federal government's pri- 
mary authority for supporting adult literacy programs, require the U,S, Department 
of Education to develop indicators of program quality for adult education programs, 
including whether they are successful in recruiting, retaining, and improving the 
literacy skills of participants. Until the results are in from this long-term project to 



Jackson reports ABE program attrition rates of 39 to 80 percent in ACBE Evaluations of 
Community Based literacy Programs (1988-89) (Washington, D.C.: Association for Community Based 
Education, 1989), p. 84. See also T. Herr and R. Halpem, Otauging Wliat Counts: Re-Thinking the 
Journey Out of Welfare (Evans ton. 111.: Northwestern University, Center for Urban Affairs and Policy 
Research, April 1991), pp. 15-16. These authors, who are conducting a long-term evaluation of 
Project Match, which serves very disadvantaged Chicago welfare recipients, question education as 
a first step, even for those with low basic skills, due to a high dropout rate in their program among 
wonvin attending GED classes. 




34 



Assessing fOBS Participants 



establish performance standards in adult education, and from evaluations that 
measure the employment-related payoffs of basic skills education, JOBS adminis- 
trators must set their own expectations for adult education for welfare recipients. At 
this time, there is a risk that assessment approaches premised on education for low- 
skilled participants will inadvertently contribute to poor program participation and 
outcomes, and possibly to low or nonexistent program. impacts as well. 
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Key Issues 

A central question for JOBS administrators about the design of a basic 
literacy assessment is: Should standardized tests be used? Related questions 
include when a literacy assessment should be done-n.e., at what point in a 
participant's program experience-and whether it should vary for different 
groups of participants. 

The principal advantages of using standardized tests for assessing basic 
literacy skills in JOBS include: 

• They are a widely available, relatively low-cost method of determining 
approximate levels of reading and math abilities, and they are valid and 
reliable when properly administered-i.e., they do measure the targeted 
basic skills, and they produce the same results time after time. 

• They are widely used to "sort" individuals into different education and 
training programs and are an established screening mechanism required 
by many education and training programs. 

Major disadvantages include: 

• They have the potential for reducing program participation by creating 
"test anxiety" among participants. 

• There is no proven correlation between the skills measured and job 
performance, even for tests that are designed to assess "functional" skills. 

• They can screen participants out of possibly beneficial activities if used as 
the sole indicator of an individual's capabilities in conjunction with strictly 
applied standards for entry into education activities. 

Less formal methods of assessing literacy skills present a trade-off as well. 
Talking to program staff and filling out program forms may be less anxiety- 
producing for participants than taking a test, but it is difficult to diagnose 
specific skill deficits or ascertain a precise grade-level equivalent of skills 
without a test. 

A question closely related to methods of literacy assessment is: What are 
appropriate program responses to JOBS participants with poor basic skills? 
High dropout rates for adult basic education and GED programs should cause 
administrators to reconsider universal, up-front testing for basic skills as an 
approach to deciding who needs education and who does not. 
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Assessing Vocational Skills, 
Aptitudes, and Interests 

Vocational assessments serve three main purposes in JOBS: (1) to identify 
occupations that participants are interested in and generally suited for; (2) to assess 
their general readiness for occupation-specific training; and (3) to guide choices 
about the specific type of occupational training appropriate for them. If participants 
are not considered imn\ediately employable and their basic skills are in the middle 
range or above (equivalent to about seventh grade or above), the likeliest JOBS 
assignments include; vocational training programs, pre-vocational education 
programs, post-secondary education programs, on-the-job training (OJT) or work 
experience placements, and career or vocational exploration activities. 

Designing a vocational assessment in JOBS is more complicated than designing 
a literacy assessment. The range of topics that can be covered is much greater, and 
many strategies are available for gathering information and using it to make 
decisions about program assignments. Like educational testing, vocational assess- 
ment is a thriving commercial enterprise with competing ideas and products. (In 
neither educational testing nor vocational assessment, however, is the market 
crowded with products or experts specialized for disadvantaged individuals.) 
Therefore, it is not surprising that, in practice, there is considerable variation in how 
welfare-to-work programs and other employment and training programs handle 
vocational assessments, or that there is little agreement about how comprehensive 
the process needs to be and where it fits in the overall program flow. 

The available vocational assessment systems or "technologies" are not clearly 
related to actual job performance. Thus, a key question for JOBS administrators 
in choosing vocational assessment approaches is whether the more complex and 
expensive systems offer additional benefits to staff and participants. If such 
systems are adopted, another question is whether vocational assessments should 
be conducted for all, or only some, participants. 

Vocational Assessment Topics and 
Information-Gathering Strategies 

The kinds of information generally considered relevant in making referrals for 
vocational training, OJT and work experience placements, and other vocational 
activities include: 

• Literacy skills and prior education 

• Prior work history 

• Occupational interests and goals 

• Knowledge of different occupations and workplace environments 

• Job attitudes and work behaviors 

• Knowledge of job-search skills and strategies 

• Vocational aptitudes 
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Some of this information comes from employability and literacy assessments 
rather than, strictly speaking, vocational assessment. An important question for JOBS 
administrators, then, is how to consolidate information gathered at different times 
or from assessments done by different agencies. Unless JOBS assessment systems are 
structured so that a vocational assessment incorporates the results of any prior 
literacy assessment, a participant can end up taking at least three different basic 
skills tests in a JOBS program: one at program entry to detemune whether she 
should be assigned to an education activity; one (or more) administered by die 
education provider to measure her progress through its program; and one 
administered by the vocational training provider to determine whether she meets 
literacy requirements for entry to training. Such a system wastes resources and risks 
losing participants because extra steps-which can look like barriers-are created in 
the program. 

Occupational Interests and Goals 

The different strategies used in employment and training programs to identify 
participants' occupational interests and goals reflect the varying philosophies about 
the decision-making process and the amount of knowledge necessary to make an 
informed decision. At the most intensive end of the spectrum are approaches that 
provide a structured vocational exploration before an employability plan is 
developed. Vocational exploration activities may include: 

• Guided research by participants on wage rates, workplace conditions, and skill 
and training requirements in different occupations; 

• Visits to work sites to see work conditions firsthand and talk to employees and 
their supervisors; 

• Presentations by employers and workers (including former program enrollees) 
with opportunities for discussion; 

• Hands-on demonstrations of workplace tools and procedures; 

• Computation of family budgets based on different wage levels; 

• Instruction and practice in job-search techniques; 

• Discussion of appropriate work behaviors; and 

• Vocational counseling. 

Vocational exploration approaches generally encourage participants to take an 
active role in acquiring information about themselves and the job worid. Also 
common are individual and group exercises that help participants realistically 
appraise their interests and options and make decisions about their career goals. The 
intent of vocational exploration is usually to "empower" participants to take 
responsibility for their employment plans and program assignments. Proponents of 
this approach argue that it increases the likelihood that participants will complete 
a training or education course and remain employed, because they are more 



49 



38 



Assessing JOBS Participants 



committed to their own (rather than program staffs) choices and to preparing for 
jobs that suit their economic needs, temperament, and interests.^ 

Such an approach was followed in a recent pilot program in New York City. The 
Employability Skills Project (ESP) enrolled welfare recipients in a course designed 
to teach them employability skills and the principles of self-directed career decision- 
making while they were active in other employment-related activities such as work 
experience and vocational training, or before they were assigned to an activity. The 
ESP course used the Career Development Series of the Adkins Life Skills Program, 
a curriculum designed to teach disadvantaged groups the attitudes, knowledge, and 
skills they need to choose an occupation and find and hold a job. It is based on a 
problem-solving approach and uses interactive exercises, role-playing, and video 
materials. ESP had multiple objectives: to help participants take positive, self- 
directed steps toward employment; to facilitate sound vocational choices; and to 
increase retention in training courses by building commitment and validating the 
participant's occupational choice. According to the pilot program researchers, the 
program was successful in increasing participants' attendance and retention in 
training as well as their job-placement rates.^* 

Another approach to assessing occupational goals and interests tests a parti- 
cipant's responses to verbal descriptions or pictorial representations of a variety of 
work settings or work situations, using questionnaires called "interest inventories." 
Typically, the test-taker indicates whether she would like, dislike, or is unsure about 
working in such conditions. The U.S. Employment Service has developed such an 
instrument; examples of commercially developed inventories include the Career 
Occupational Preference Survey and the Jackson Vocational Interest Survey. 

A simple approach relies on participants' expressed preferences for certain types 
of jobs. Often, although not always, programs in which staff have high caseloads use 
participants' occupational or training choices as the main guide to vocational activity 
assignments, as long as their education, work history, and basic skills levels suggest 
that they can meet entry requirements established by local training providers. 
Programs that subscribe to the "empowerment" philosophy may also give great 
weight to client preferences in vocational assessment decisions; their guiding 
principle is "Let them try it out." 

In programs that neither conduct formal testing to identify job interests nor 
provide a structured career exploration unit, one-on-one interviews are usually the 
main method of vocational assessment. A typical interview takes 45 minutes to VA 
hours, covers all the topics listed above for vocational assessment, and results in an 
employability plan. Such an approach relies more on what participants already 
know about job opportunities and careers than do formal approaches that include 
tests. 



^^elephone interviews with Cynthia Morano, Executive Director, Wider Opportunities for 
Women, Washington, D.C., and John Doerrer, Senior Vice President, Training Development 
Corporation, Bangor, Maine. 

^M. Bridges et al., Employability Skills Project: A Life Skills Approach to Making Welfare Work (New 
York: New York City Department of General Services, August 1988). 
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Vocational Aptitudes 

A longstanding interest among employment and training professionals in 
improving retention and completion rates in vocational training courses, and 
retention in employment, has stimulated an aptitude assessment industry. Two types 
of instruments have been developed. One assesses job-related skills through pencil- 
and-paper tests supplemented with a few tests using physical apparatus. The other 
type measures job-related aptitudes in terms of an individual's performance on 
"work samples" or at "work stations" that simulate a work environment. 
Comprehensive assessment systems using computerized analysis combine the two 
approaches. 

As an aspect of vocational assessment, aptitude testing has not played a 
prominent role in JOBS programs, although it remains central to vocational 
rehabilitation programs and is prominent in the JTPA system. One of the key issues 
for JOBS administrators is evidence: Few relationships between the aptitudes 
measured and actual job performance have been demonstrated through research that 
controls for background differences among the people studied. Furthermore, the 
interpretation of aptitude test results is a special area of expertise not always found 
in employment and training programs, and converting test results into decisions 
about appropriate local vocational training, work experience, and other services 
requires the judgment of knowledgeable staff. 

Pencil-and'Paper Tests. The G ATB (General Aptitude Test Battery), developed 
by the U.S. Employment Service in 1947 and revised in 1983, is one of the most 
widely used vocational aptitude tests for employment counseling and referral. The 
test, which takes hours to administer, measures nine traits: intelligence, verbal 
" aptitude, numerical aptitude, spatial aptitude, form perception, clerical perception, 
motor coordination, finger dexterity, and manual dexterity. Most of these are tested 
with pencil and paper; a few require the manipulation of physical apparatus, for 
example, inserting pegs into holes or assembling and disassembling small washers 
and rivets on a board. 

The skills analysis for the GATB is based on the U.S. Department of Labor's 
Dictionary of Occnpatimial Titles (DOT), which is a widely used tool for matching 
workers witin jobs.^^ The most recent version of the DOT classifies 12,(X)0 jobs 
according to a scheme of broadly defined performance requirements. 

In 1986, the U S. Employment Service announced plans to make the GATB the 
sole basis for job referrals from that agency and to expand the range of jobs for 
which it was considered a valid predictor of success— from the original 500 to 
virtually all occupations in the U.S. economy. The announcement prompted legal 
challenges, followed by a full-scale review of the test by a committee of experts 
appointed by the National Academy of Sciences. A U.S. Department of Labor policy 
suspended use of the GATB as the sole basis for screening and referrals to 



^^J. Hartigan and A. Wigdor, eds., Fairness in Employ)nent Testing: Validity GeneraliTation, Minority 
Issues, and the Getieral Aplitude Test Battery (Washington, D.C.: National Academy Press, 1989), pp. 
60, 63, and 135. 
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employers by the U.S. Employment Service. (It may still be used for Employment 
Service counseling on a voluntary basis.) 

The results of the study should serve as a caution to organizations considering 
using this test and similar tests in their JOBS assessment. The judgment of the expert 
panel was that "GATE scores can provide useful screening infomxation, but the 
predictive power of the test battery is not so strong that the GATB should become 
the sole means of filling all job orders."^ The conunittee also warned that the 
proposed referral policy would have the effect of reducing opportunities for 
minorities because the test would screen out a large proportion of minority job 
applicants who would be capable of performing well on the job. The panel's 
technical review was also skeptical of claims about the generalizability of the test 
results to the full range of occupations in the economy; they advised caution in 
applying the test results beyond tlie original range of occupations for which it was 
developed .^^ 

Tests that are similar to the GATB and widely used include the Armed Services 
Vocational Aptitude Battery (ASVAB), the Differential Aptitude Test, the Employee 
Aptitude Survey, and the Wonderlic Personnel Test. The panel that studied the 
GATB rated it against these other tests and found that they offered little, if any, 
improvement over the GATB in their ability to predict success in training.^® 

Work Station Testing. "Work station" or "work sample" tests are alternatives to 
pencil-and-paper testing methods. Initially developed for programs serving the 
mentally or physically handicapped, they were adapted for economically and 
culturally disadvantaged individuals in the 1960s. Under U^e auspices of the U.S. 
Department of Labor, the Jewish Employment and Vocational Services (JEVS) of 
Philadelphia pioneered these testing methods in programs for the disadvantaged. 
The most recent JEVS product is known commercially as VITAS (Vocational Interest, 
Temperament, and Aptitude System). It includes 21 work samples, in each of which 
the participant performs a task similar to some aspect of a job. The range of work 
samples, which takes 216. days to administer, includes clerical, mechanical, technical, 
and physical tasks tliat are considered to be related to thousands of job listings in 
the DOT. Tlie speed and accuracy of the work performed determine a person's 
score. 

VITAS also provides information on individuals' vocational interests and 
temperamental suitability for certain types of jobs. The test-taker's reactions to the 
work samples, based on behavior during the test and on a personal interview, are 
the measures of vocational interests. "Work-related temperaments"-5uch as a 
tolerance for repetitive tasks, an ability to plan and organize, and an ability to 
cooperate with others— are judged by an evaluator who observes the testing process. 
An assessment report synthesizes the results of the work sample tests and 
observations, and identifies job classifications in the DOT and the U.S. Department 



^Hartigan and Wigdor, p. 5; see also Chapter 8. 
^^Hartigan and Wigdor, p. 8 and Chapters 6 and 7. 



^Hartigan and Wigdor, pp. 91-98. 
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of Labor's Guide for Occupational Exploraticm for which the participant has 
demonstrated potential. 

The developers of work station tests have traditionally claimed two main 
advantages of this type of assessment: It is useful for populations that cannot read 
well enough to take traditional pencil-and-paper tests, and it is not likely to generate 
the same type of test anxiety as a written test. The testing equipment is expensive, 
though, and a limited number of participants can be accommodated at one time. 

Comprehensive Vocational Assessment Systems. Some commercially developed 
assessment instruments use computer systems to integrate the information-gathering 
and decision-making functions in a vocational assessment. Typically, they offer a 
battery of tests of vocational aptitude, occupational interests, and basic literacy skills. 
In addition to the individual's scores on each test, the computer-generated 
evaluation report provides a vocational profile of the individual. It recommends 
specific training or jobs that seem appropriate by matching a test-taker's results to 
databases containing information about occupation-specific competency 
requirements. 

Another JEVS product, APTICOM, is a widely used example of this approach. 
The vocational aptitudes section of APTICOM contains 11 tests measuring 10 
aptitudes; it closely resembles the design and format of the G ATB. The occupational 
interest inventory section uses the 12 interest areas defined in the Guide for 
Occupational Exploration and is derived from the interest inventory instrument 
developed by the U,S. Employment Service. Test-takers indicate whether they like 
or dislike descriptions of work tasks and environments, or if they are undecided. 
The academic skills section measures language and math ability based on the scales 
of the GED test. 

The entire battery takes between IVi and 2 hours to administer, considerably less 
time than the longer versions of the tests from which it is derived. Because it is 
computer-based, results are available almost immediately. Tlie final report, in 
addition to including scores on the individual tests, identifies occupational groups 
and examples of specific jobs (based on DOT listings) tliat are believed by 
APTICOM's developers to be appropriate for the test-taker. 

The technical soundness of the APTICOM battery has been rated as good as or 
better than that of the longer tests.^^ However, the ability of the APTICOM system 
itself to predict training outcomes or on-the-job performance has not been tested. 
Also, because it uses GED-like items, APTICOM is more academic in tone than 
CASAS tests or the TALS. The system is designed to reduce the anxiety and 
boredom associated with the more traditional tests, but whether it does this rennains 
to be determined with certainty. 

KEVAS and JOBTAP are other commercially developed, computer-based tests 
that autonrtatically process information about an individual's vocational aptitudes, 
interests, and academic skills to predict her success in specified types of 
occupational training- KEVAS (Key Educational Vocational Assessment System), 



•^arl F. Botterbusch, "Vocational Assessment and Evaluation Systems: A Comparison," reprinted 
in Vocational Research Institute marketing materials. 
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developed by Key Education, Inc., in Shrewsbury, New Jersey, includes tests that 
measure perceptual and motor skills, abstract reasoning and problem-solving ability, 
vocational interests, "temperament factors" considered important to job performance 
(social functioning, res|X)nse to stress, work attitudes), and reading and math skills. 
Background and demographic characteristics of test-takers are collected in a 
structured interview and entered into the automated system by program staff. The 
KEVAS database also includes an "occupational bank" containing over 600 
occupational profiles developed from, the DOT and other sources. Test results and 
background information are matched by computer to the occupational profiles in 
order to generate recommendations for occupations with skill requirements that are 
in keeping with a participant's tested capabilities and interests. The computer- 
produced KEVAS evaluation report also makes recommendations for skills and 
attitudinal areas that need improvement. The KEVAS battery takes about 2Vi hours 
to administer. 

JOBTAP (Job Training Assessment Program), an automated assessment tool for 
vocational education and training programs for the disadvantaged, takes a 
somewhat different approach. Developed by the Educational Testing Service in 
Princeton, New Jersey, it is designed to help service providers determine who 
among the applicants to their programs would be most likely to succeed in tlie 
available training courses, as well as to help individuals find appropriate services. 
JOBTAP includes checklists for an individual's vocational interests and work history, 
and tests a spectrum of work skills and job-related basic skills. The computer- 
generated report evaluates the individual's performance against a nationally 
representative group, produces a measure of her likelihood of success in the type 
of training offered by the provider, and makes recommendations for a short-term 
career goal, training program, and long-term career goal. By comparing the scores 
of all applicants to the same training courses, training providers can determine who 
are most likely to succeed. 

Workplace Knowledge and Job-Search Skills 

Although participants' understanding of workplace attitudes and behaviors and 
of job-search techniques are often assessed as part of a review of their job-readiness, 
some program administrators believe that participants can be better advised about 
the type of vocational training they should enter if these traits are considered in 
vocational assessments as well. Tests have been developed for this purpose. PA YES, 
the Program for Assessing Youth Employment Skills, also developed by the 
Educational Testing Service, is specifically for young adults and youths with low 
verbal skills. This three-part test measures understanding of acceptable work 
behavior, attitudes toward supervision, and self-confidence; knowledge of different 
types of jobs, job-search skills, and ability to follow directions; and interest in 
different occupations. PAYES takes about 75 minutes to administer. This type of 
assessment is currently a major focus in the JTPA system, since youth participants 
in JTPA programs must demonstrate "pre-employment" competencies in these areas. 
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Personality Tests 

Personality tests are often used with the intention of measuring an individual's 
level of self-esteem. They may also provide information about how the test-taker 
views her interactions with others. In these tests, individuals respond to statements 
or questions about traits or behaviors, indicating whether they are accurate 
descriptions of themselves. Personality tests that have been used in welfare-to-work 
or employment and training programs or are generally considered to be of a high 
technical quality are: the Jackson Personality Inventory QTl), the California 
Psychological Inventory (CPI), the Coopersmith Self-Esteem Inventory (SEI), and the 
Culture-Free Self Esteem Inventories. It should be noted, however, that such tests 
generate hostility and anxiety in some test-takers.^ 

Critics of personality tests generally believe that they do not accurately predict 
success on the job and may be culturally biased, personally intrusive, or conducive 
to negative stereotyping. Supporters say that they provide insight into traits that 
affect how participants relate to coworkers, supervisors, and different work 
situations. 



ConsideratioiiS in Designing a JOBS 
Vocational Assessment 

JOBS administrators who decide to invest significant sums of money and time in 
vocational testing expect the resulting program assignments to be better than if 
participants and staff simply used their own experience and common sense, perhaps 
guided by some broad criteria. They also do not want to "reinvent the wheel." 
Employment and training professionals have been updating and refining their 
knowledge of occupational skills for decades, and an extensive literature addresses 
the relevant issues in labor supply and demand. However, the relationship between 
the measurable characteristics of disadvantaged individuals and their performance 
in vocational training programs is unknown. Furthermore, the relationship 
between vocational training and success in the job market has not been rigorously 
measured. The evidence that is available about the effectiveness of some programs 
(not including vocational training, for the most part) has not been integrated with 
available knowledge about targeting program services to certain groups; therefore, 
a picture of what works best for whom cannot be drawn for disadvantaged 
individuals.*^ Existing information about targeting services to groups of clients 



^^ackson. Measures for Adult Literacy Programs. 

*'For a discussion of the problems in comparing the effects of programs targeted to different 
groups of welfare recipients, see Gueron and Pauly, From Welfare to Work. An evaluation now 
underway of employment and training services provided to low-income adults and youths through 
the JTPA systcnt-sponsored by the U.S. Department of Labor-promises to expand considerably the 
evidence available about the effects of vocational training for specified groups of the eligible 
population, including welfare recipients. 
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consists largely of descriptions of entry requirements, sometimes accompanied by 
outcome data,*^ 

For JOBS administrators, this means that practical considerations may be most 
important in their decisions about vocational assessment. As in other areas of JOBS 
assessment, there are trade-offs. For example, on a fixed budget, JOBS programs 
may not be able to afford in-depth vocational assessments for all participants. 
Administrators may need to choose between providing simple assessments for all 
participants and extensive assessments for some-perhaps done after some 
participants have been assigned to other activities or left the welfare rolls for work. 
They may opt to conduct vocational assessments only for participants whose initial 
screening shows that they are eligible for selected vocational training programs, in 
order to document skills required for entry to certain vendors' programs. Or they 
may target vocational assessments to participants who are uncertain about their 
vocational goals and are assigned to a vocational exploration activity. Some program 
administrators employ vocational assessments primarily as an educational tool for 
participants, recognizing that their predictive power is limited. 



^^See Yudd and Nightingale, Vie Availabilily of Infonnation far Defining and Assessing Basic Skills 
Required for Specific Occupations. 
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Key Issues 

A critical question in designing a vocational assessment for JOBS is: Do the 
specialized tests available improve the ability of staff and participants to 
make appropriate choices about program activities enough to warrant the 
additional expense and time that they require? 

There is little evidence from research to answer this question, and only 
very limited information to indicate whether such tests can accurately predict 
who will be successful in training or on the job. What is clear from an expert 
review of the General Aptitude Test Battery (GATB), widely used in vocational 
assessments, is that there are significant concerns about using this test, as well 
as similar tests, as the sole basis for determining whether job applicants can 
perform adequately on the job. People who score low are frequently able to 
perform jobs they would have been excluded from on the basis of the test. 

Operational experience with vocational assessment approaches is mixed. 
Some staff and programs respond enthusiastically to specialized testing 
strategies, while others question the value of specific types of tests-in 
particular, interest inventories and personality tests. Thus, important 
considerations for JOBS administrators are: 

• Whether vocational assessment is related to local training options and job 
opportunities. If the latter are limited, the value of extensive vocational 
testing is questionable; similariy, computer-based assessment systems that 
recommend occupational options not locally available are not helpful to 
participants and may waste resources. 

• Whether, to conserve resources, it is better to cover some vocational topics 
early and-for those who need it-other topics later, instead of scheduling 
all participants for a comprehensive, intensive assessment when they enter 
the JOBS program. 

• Whether to integrate some parts of a vocational assessment into other JOBS 
activities-for example, into vocational exploration, training programs for 
workplace attitudes and behavior, and job-search workshops. 

• Whether to require consistent assessment procedures across a state or to 
allow locaHties flexibility in selecting the agency responsible for assessment 
and the specific strategies and instruments used— based on variations in 
local service provider practices and labor markets, for example. 

• Whether to establish specific criteria for referring participants for in-depth 
assessments or job training, or to encourage case-by-casc decision-making 
on the basis of staff and participant choices. 
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Assessing Social Service Needs 

The purpose of a social service needs assessment in a welfare-to-work program 
has traditionally been to identify personal, family, or situational problems that could 
interfere with a participant's attendance in program activities, job retention or 
performance, or potential for self-sufficiency. Such assessments have usually focused 
on transportation and child care arrangements, participants' physical and emotional 
health, and housing and legal problems. In JOBS, this area of assessment combines 
traditional social work practice with the program's employment focus, as was the 
case in the earlier Work Incentive (WIN) Program. 

The JOBS legislation breaks new ground, however, by giving states the option of 
also assessing the needs of children when program staff review family situations 
that might affect the parent's (generally the mother's) program participation or 
employability. This provision has the potential to greatly expand the role of the 
JOBS agency in identifying and addressing the social service needs of welfare 
families,^^ although there is little operational experience-and even less research 
evidence— to guide agencies in this area. 

The key question for most JOBS administrators in designing social service 
assessments is whether to go beyond existing approaches^i.e., to also identify the 
needs of participants' children, as the law now allows, and to respond to the 
needs of participants and their children with social services-rather than 
emphasize policies that excuse those with personal problems from participation 
in employment services. Advocates of "doing more" point to the multiple problems 
of the most disadvantaged welfare recipients and the reduced life chances of their 
children, arguing that employment services alone are unlikely to be the answer to 
poverty.^* 

Problem Identification 

The methods used to gather information in social service needs assessments may 
determine how often, when, and what types of personal problems are identified 
among participants in JOBS programs. These methods range from a checklist of self- 
reported information to home visits. One assessment instrument develop^ for a 
welfare employment demonstration program included a checklist covering eight 
areas-child care, transportation, housing, drug and alcohol problems, personal 
attributes (such as attitude, appearance, and behavior), legal problems, health, and 



*^See O. Golden et al. Welfare Reform and Poor Children: Collaboration and Case Management 
Approaches, draft (Cambridge, Mass.: Harvard University, John F. Kennedy School of Government, 
September 1990); and S. Smith, S. Blank, and R. Collins, Pathzvays to Self-Sufficiency for Tzoo 
Generations: Designing Welfare-to-Work Programs That Strengthen Families and Benefit Children (New 
York: Foundation for Child Development, 1992). 

**See Smith, Blank, and Collins, Patlnvays to Self-Sufficiency for Two Generations, and Zill et al, 
Yielfare Mothers as Potential Employees. 
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language barrier^and some questions to help staff to identify whether a problem 
existed.*^ Based on the client's response, a program staff member indicated on the 
assessment form whether a problem existed, whether it was severe enough to be a 
barrier to participation, and whether there might be a problem in the near future. 
The checklist was designed to be completed in 10 minutes. 

An alternative method is assessment of social service needs by direct observation. 
Some programs conduct lengthy, inniepth assessments requiring several days or 
weeks of activities in order to observe participants firsthand. Staff members look at 
patterns of attendance, behavior, and attitudes to identify underlying problems that 
may need to be addressed before participants enter a program component.^ Staff 
members in some programs may visit parents and children at home or observe 
children in a child care center. (Both of these methods of assessment are rare,*^ and 
the latter tends to be done only if the program itself provides on-site child care.) 

More common than checklists, in-depth assessments, or home visits are problem 
identification approaches that rely on initial interviews or direct observation of 
clients in their first program activity. The extent of such assessments differs 
depending on whether interviewers probe and explore the information provided by 
participants. For example, when inquiring about a participant's child care 
arrangements, an interviewer can explore her preferences and concerns, educate her 
about different types of care, and discuss the options that might be most suitable for 
her and her child. Current practice also varies depending on: 

1. Tlte responses available, Wl^ere services for some common but difficult social 
problems are scarce— such as slots in treatment programs for substance abuse and 
assistance with housing relocation or costs— programs may identify but not 
explore these needs. Participants with these problems rmy be routinely excused 
from participation. Where JOBS programs cooperate with service organizations 
that can respond to needs that extend beyond income and employment, there is 
a higher probability that JOBS staff will attempt to uncover these needs. 

2. Tlte resources available for assessment Where the number of assessment staff, 
their time, or program funds are limited, JOBS programs may rely on participants 
to identify their own needs. Tl^ey may also adopt an "exception" strategy of 
exploring social service needs if participants fail to attend assigned activities. 
Where program intake is slow enough, assessment caseloads are low enough, or 
special assessment services can be purchased easily, JOBS programs may adopt 
an intensive assessment approach that includes a study of family and children's 
needs. 



*^he instmment was developed by the Urban Institute for the Pennsylvania Saturation Work 
Program. For a discussion, see D. Nightingale, Assessing the Employ ability of Welfare Clients 
(Washington, D.C.: Urban Institute, May 1986). 

^^wo programs that conduct such assessments are operated by Wider Opportunities for Women 
in Washington, D.C., and the Training Development Corporation in Bangor, Maine. See below for 
further descriptions of these programs. 

*^See Golden et al.. Welfare Reform and Poor Children. 
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3. The philosophy of the program. Programs that emphasize labor force attachment 
may give information about the social services available and expect participants 
to identify their own needs. Programs that aim to maximize participation 
son\etimes take this approach as well, in order to avoid focusing on reasons that 
eligible individuals may have rwt to participate. A program that emphasizes 
human capital development, and has staff trained in human behavior, is more 
likely to assess individual and family functioning as an initial step. 

Responses to Social Service Needs 

In the past, there have been three main responses in welfare employment 
programs if staff uncovered problems or situations likely to interfere with an 
individual's ability to participate in the program or take and keep a job: 

1, If a parent had a problem covered by standards for exemption from participation 
in the program, she could simply be excused, or she could be referred to a social 
service provider. Under JOBS, these are still possible responses. 

2, If the parent had "good cause" for not participating, according to state or local 
program policies, she could be excused until the problems were resolved. This 
entailed additional decisions about how long such a "deferral" was to last; what, 
if any, services were to be provided in the interim; and who would provide the 
services. Under JOBS, this is also a possible response, 

3, If the parent neither met exemption criteria nor had "good cause" for 
nonparticipation, but the famil/s or her problems were nevertheless likely to 
make participation in the program or holding a job difficult, welfare employment 
programs of the past usually had few responses. They could arrange and/or pay 
for child care and transportation, provide individual counseling, make referrals 
to social service agencies, or make payments for "special needs." Under JOBS, the 
program can meet social service needs either directly or in coordination with 
other agencies. Tlie family problems and tlie service responses are not specified 
by the law. 

In setting policy and developing criteria and procedures for assessment of the 
social service needs of JOBS participants (but not tlieir children), JOBS adnrdnis- 
trators have extensive experience to draw on from WIN and WIN Demonstration 
programs. JOBS does not use either the formal designation of "deferral" for eligible 
(nonexempt) welfare recipients who are temporarily unable to participate, or the 
"appraisals" of social service needs that were central to the WIN program 
registration process. There are similar practices in JOBS, tliough; many programs 
have modified "front-end" WIN appraisal methods to fit new JOBS terminology and 
assessment procedures. 

Thus, in some JOBS programs, acceptable reasons for deferring participants from 
program activities have been established. Staff members review these with each 
participant during an initial interview and make referrals or arrangements for social 
services on a case-by<ase basis. In other programs, participants are informed about 
program activities in a group orientation, and they meet with a staff member for 
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discussion of their social service needs only if they identify circumstances that 
would hinder their program attendance. Some JOBS programs have established 
standard social service "packages" for participants, often including vouchers for child 
care (which participants find themselves, guided in many locations by child care 
resource and referral agencies, or lists of child care providers from JOBS staff) and 
transportation allowances. These services are sometimes supplemented by others, 
including funds for special needs that participants and staff identify during 
assessments. 

Assessments of JOBS participants' social service needs that go beyond child care 
and transportation are likely to uncover problems that traditionally have not been 
a focus of welfare-to-work programs. Because the needs of children are inextricably 
linked to those of their parents, the new options in JOBS can be interpreted as a 
broad opportunity to identify and respond to problems like parental substance 
abuse, domestic violence, and poor parenting behavior, as well as to children's 
physical health and development. JOBS administrators need to consider whether to 
devote program resources to these needs, either directly or through agreements with 
service providers. 

If JOBS programs rely primarily on referrals to other organizations for social 
services, the referral process may need attention. For example, will staff members 
simply inform participants about the availability of services and let them make their 
own arrangements, or will they actively negotiate services for individual JOBS 
participants? Will they follow up with participants and agencies? Will they intervene 
in certain circumstances? 

The Timing of Social Service Assessments 

The employability characteristics of JOBS participants and their vocational 
aptitudes and interests tend to change slowly (if at all) as an individual progresses 
through a program. Social service needs are more dynamic. Because circumstances 
change, arrangements break down, and unforeseen difficulties emerge, social service 
assessment or monitoring capability generally needs to be available continuously, 
not just when initial program assignment decisions are made. This requires 
communication between the JOBS staff responsible for seeing that such needs are 
met and staff in charge of die activities in which participants are enrolled, if these 
responsibilities are separate. In JOBS programs that provide case management, the 
case manager can monitor both clients' social service needs and their participation 
in employment-related activities. Thus, social service assessment intersects with the 
broader issues of case management, client activity tracking, and inter-agency 
communication. 

Frequent contact between case managers or other program staff and participants 
to continually monitor their social service needs also has a particular advantage: 
Because the circumstances addressed in a social service needs assessment are 
sometimes highly personal, developing participants' trust in staff is essential, a 
process that often takes time and continuity of relationships. 
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Key Issues 

Welfare-to-work programs have rarely had the capability to address the 
sodal service needs of vs^elfare families beyond the supports required for 
clients to participate in employment-preparation activities, such as child care 
and transportation. The JOBS legislation gives states the option of assessing 
the needs of children in families with adults eligible for JOBS programs, which 
opens up new opportunities and challenges for program planners. Little 
research and few models are available to guide decisions in this area. 

Topics of interest to JOBS officials include: 

• Methods of problem identification. Simple checklists that rely on 
participants to flag their own concerns, interviews of varying intensity, 
observations of participants in program activities and of their children in 
child care, and home visits can be used to evaluate child and family 
functioning. As in educational and vocational testing and assessment, there 
are few sodal service assessment tools that reliably predict how well 
children or families will fare if they receive a particular type of service or 
no service. Thus, JOBS administrators may want to develop strategies to 
screen and prioritize families for intensive sodal service assessment. In the 
same way that intensive vocational assessments in JOBS might be reserved 
for partidpants being considered for costly vocational training, intensive 
sodal service assessments might be reserved for families being considered 
for costly sodal service interventions. 

• Responses to social service needs. In the past, welfare employment 
programs usually deferred or exempted a large proportion of theoretically 
eligible individuals because of temporary or long-term '^barriers to 
employment." JOBS programs are permitted to provide a wide range of 
sodal services to eligible adults and their children to improve families' 
prospects of self-sufficiency, although so far this seems to consist mainly 
of referrals to existing social service programs. JOBS assessment staff may 
need to expand the types of problems they screen for to include all those 
for which the program or locality has a service response. 

• Targeting and staging assessments. Like assessments of job-readiness, basic 
literacy skills, and vocational interests and aptitudes, assessments of JOBS 
participants' social service needs can be designed to vary by subgroup 
within the eligible population. High-risk groups—for example/ teenage 
mothers— might be targeted for more intensive social service evaluations. 
However, unlike other types of assessment that are intended primarily to 
lead to assignment to a JOBS component, assessment of sodal service 
needs is an ongoing process. Problems can emerge at any time that have 
the potential for interfering with program partidpation or employment. 
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Program Examples: Variations in 
Assessment Strategies 

The welfare employment programs of the 1980s illustrate many of the approaches 
to assessment that JOBS administrators can adopt, as well as the ways that program 
objectives and designs interact with assessment. In programs that emphasized staff 
and client choice, or employed experienced vocational counselors, determinations 
of immediate employability were usually made on a case-by-case basis, using 
subjective criteria such as a participant's motivation and preferences as well as more 
objective ones such as work experience and education level In these programs, 
staff knowledge of the local labor market and their accumulated experience working 
"With welfare recipients allowed them to judge the employability of individual 
clients. Even in programs that prescribed services on the basis of participants' 
characteristics or circumstances, staff discretion and program philosophy played an 
influential role in determining "who went where." 

In the Baltimore Options Program, which was designed to serve 1,000 WDM- 
eligible clients at a time and operated between 1982 and 1986, there were general 
guidelines for "who should go where," but particular decisions depended a great 
deal on what participants wanted to do, and on whether staff thought they would 
benefit from the service option they selected. The program offered participants a 
range of activities when they enrolled. Staff relied on the results of a one-hour test 
of reading and math skills, information from forms on which participants recorded 
their prior education and work history, and a 45- to 90-minute interview during 
which participants' interests, goals, family situations, and supportive-service needs 
were discussed. Participants who wanted vocational training and met the education 
criteria for entry were referred to training vendors. Any additional vocational 
assessment was done by these vendors. Overall, this relatively unstructured 
assessment approach resulted in approximately 25 percent of eligible individuals 
participating in a job-search activity, 17 percent in work experience, and 16 percent 
in education or training.^^ 

Other pre-JOBS welfare employment programs defined employability according 
to objective criteria, which staff used to enroll participants in components when they 
entered the program. Most often, all eligible individuals who met a test of 
employability were assigned to job search. For example, the policy of the New 
Jersey REACH Program, which began in 1986 and has changed little under JOBS, 
describes an individual as job-ready if she has worked in a job that pays above 
minimum wage for at least six months during the previous two years; has worked 



^^taff and client choice usually played a smaller role in decisions about immediate employability 
in fixed-sequence models that began with job search, though participants' preferences may have 
been important to decisions about education, training, and other subsequent activities. 

See J. Quint, Maryland: Interim Findings from tJie Maryland Employinent Initiatives Programs (New 
York: MDRC, 1984). 
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in an occupation that is in demand in the local labor market; or has recently 
completed vocational training.^ 

Assessment in REACH, a program that is mandatory for welfare recipients with 
children at least two years old and that is usually operated by county welfare 
departments, highlights the role of local variation and client choice in welfare-to- 
work programs. REACH participants go through a quick screening process when 
they enter the program. It focuses on their work experience, employment goals, and 
English-speaking ability. Individuals who are not assigned to job search or ESL 
classes as a result of the screening are referred to another organization for an in- 
depth vocational assessnnent that includes a formal test of basic skills. Aside from 
the criteria for job-readiness described above, the state has not established criteria 
for assigning individuals to specific JOBS activities in this program. 

Each local REACH program is allowed to choose the agency to conduct the 
REACH assessment. Most localities leave decisions about specific instruments, 
strategies, and topics to the assessment agency selected. As a result, assessments 
vary considerably across the state. In some counties, tlie agency responsible for 
assessment tests basic skills (often using TABE or ABLE) and vocational interests, 
and leaves additional testing to training providers, Otlier counties require a battery 
of vocational aptitude tests, interest inventories, and personality tests. Some 
assessments take only a few hours; others, several days. 

The Pennsylvania Saturation Work Program (PSWP) was a demonstration 
project operated between October 1985 and September 1987 in two sections of 
Philadelphia. It was designed to test the feasibility and effectiveness of a welfare-to- 
work program focused on high participation rates (the "caseload saturation" 
concept). PSWP designated participants as immediately employable if they had 
completed at least the tenth grade or had a GED a77d had been employed for six 
months during tlie previous two years or had a current job skill,^' To systematize 
the assessment process, the Urban Institute developed a data-collection instrument 
that served both to document these employability characteristics and to guide staff 
choices for program assignments. After tlie initial determination of participants' 
potential for immediate placement in a job, participants took a short (48-minute) test 
that assessed their reading and math levels, knowledge of work and job-search 
techniques, and occupational interests. Job-ready participants were referred to 
independent job search. Based on the test results and an individual interview, staff 
assigned the others to basic education, employment workshops, or vocational 
training. 

Although PSWP issued guidelines rather than hard-and-fast rules for activity 
assignments, it emphasized quick job placement. However, staff tended to refer 
participants to basic education programs rather than job search if they had not 



^^^See "REACH Policy and Planning Update #6," issued by the New Jersey Department of Human 
Services in 1987. Prior to implementation of JOBS, "eighth-grade reading and mathematics ability" 
was also considered to be evidence that a REACH participant had job skills and experience 
applicable to the local labor market. 

^'S. Hogarth, R. Martin, and K. Nazar, Pennsiflvania Saturation Work Program: Impact Evaluation 
(Harrisburg, Pa,: Pennsylvania Department of Public Welfare, October 15, 1989). 



Program Examples: Variations in Assessment Strategies 



53 



completed school, even if tliey had some job experience. Individuals for whom 
vocational training seemed appropriate (on the basis of the screening test) were 
referred to training providers, who then did the same type of in-depth assessment 
as for any applicant to their training programs. 

Assessment begins with a standardized test in California's Greater Avenues for 
Independence (GAIN) Program, which began in 1986 and has continued to operate 
with some modifications as a mandatory statewide JOBS program. During orienta- 
tion, all English-speaking participants take a reading and math achievement test. 
Those with scores above a specified level who have a high school diploma are 
assigned to job search. The rest have the option of enrolling in basic education (or 
ESL) or an immediate job-search activity.^^ The initial GAIN assessment and 
assignment process is quite systematized and based on prescribed criteria. In-depth 
vocational assessments are conducted only for participants who complete an initial 
job-search activity without success, or who complete an education program followed 
by job search and are still unemployed. 

As noted in Chapter 4, GAIN'S testing instrument is the California Adult Student 
Assei^sment System (CASAS). But while testing is uniform, actual practice in making 
program assignments varies from county to county. In some counties, all GAIN 
participants who are determined to need education on the basis of their CASAS 
scores are strongly encouraged to choose this option (instead of job search); in one 
county, a local quota for entries to employment encourages GAIN staff to downplay 
the education option for clients who score low on the appraisal test; in others, the 
choice between education and job search is made more variably. 

In GAIN, responsibility for in-depth vocational assessment is contracted out by 
each county and the vocational assessment is done only after participants have 
completed education or job search. The procedures followed by the JTPA agency in 
one county illustrate just how lengthy and intensive such assessments can be. In this 
local program, the vocational assessment begins with a four-hour group session. It 
includes two standardized tests-the Career Occupational Preference Survey (COPS), 
which is an interest inventory, and the Career Ability Preference Survey (CAPS), 
which measures reading ability, perception of spatial relations, manual dexterity, 
and math skills— and group exercises in which participants talk about themselves 
and read aloud. 

In this county, these activities are followed by two-hour individual sessions with 
staff to discuss each participant's employment and education background, living 
situation, family issues, economic situation and needs, and career options test 
results. Participants who have no dear idea about what they want to do are 
scheduled for additional tests, but these individuals are few, according to staff. The 
discussion is intended to educate clients about career options and the requirements 
of different training programs. At the end of their interviews, participants are told 
to think about their options, discuss them with friends and family, visit some 



Riccio et al., GAIN: Early Implaiientation Experiences and Lessons (New York: MDRC, 1989). 
^^See Riccio et al., GAIN: Early Implementation Experiences and Lessons, and S. Freed man and J. 
Riccio, GAIN: Participation Patterns in Four Counties (New York: MDRC, 1991). 
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training centers, and return in a week for another two-hour session to decide on the 
recommendations that will go to the GAIN staff. 

In another California county, where in-depth vocational assessments are done by 
a vocational training vendor, GAIN participants also take the COPS and CAPS tests. 
In addition, at the discretion of the staff, they generally take a personality test and 
an achievement test measuring basic skills in grade-level equivalents. Participants 
typically lake each test on a different day; each time they come in, they meet 
individually with staff. These discussions cover their employment history, education 
background, financial situation, and family issues. Participants are also asked to visit 
training vendors and to talk with teachers and employers about the job market and 
the training required for different occupations. Although the testing itself takes only 
six to eight hours, the assessment process typically takes up to two months. 

Two other programs illustrate additional variations on in*depth vocational 
assessment. In some locations in Maine's JOBS program, called ASPIRE, a 
structured, in-depth vocational assessment is conducted by the JTPA agency (the 
Training Development Corporation) at the start of program participation. The 
process takes several weeks and includes formal testing to determine the 
individual's vocational aptitudes (using APTICOM), reading and math skills (the 
full-length TABE), and job interests; three days in a career exploration workshop 
(activities include researching the local labor market and the skill requirements and 
work environments of different occupations); and interviews with staff that result 
in a p>ersonality profile. The emphasis throughout the assessment, according to staff, 
is on "client empowerment"— helping the women to make responsible decisions about 
their lives and futures. 

Wider Opportunities for Women (WOW) of Washington, D.C., operates a wide- 
ranging, intensive, structured assessment lasting two to three weeks (three to five 
hours a day). Participants in WOW are minority single mothers between ages 19 and 
35, and most receive AFDC. The assessment is designed to help the women choose 
an education or training program and to expand their sense of what they can do. 
It focuses on career exploration to show that numerous options are available, 
including nontraditional jobs that pay better than many traditional ones. As part of 
the assessment, participants are taken to work sites to see actual job conditions and 
given opportunities to meet with employers and employees in a variety of 
occupations. WOW does not do fornnal testing to determine vocational aptitudes or 
interests; program staff believe that the job market is changing so rapidly that such 
tools are of little help to the participants. But they work with the women to 
document their work experience, including volunteer activities as well as paid 
employment. Literacy testing is an important part of the assessment, but it is not 
done at the beginning of the program because of staff concems about test anxiety. 

Project Match, a voluntary welfare employment program for long-term welfare 
recipients who live in public housing in Chicago, takes an approach to basic skills 
assessment tliat is consistent with its small scale and emphasis on client choice. 
Project Match does no forma! testing but uses informal, staff-intensive strategies to 
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evaluate participants' basic skills and their relevance to employability.^* To judge 
basic skills, staff members observe participants' speaking ability during enrollment 
interviews, learn about their personal backgrounds, and analyze the forms parti- 
cipants fill out during intake and assessment. Staff members also call participants' 
sdiools to talk with former teacliers, both to verify information provided by the 
participants and to gain a better understanding of their experiences and attitudes. 

Project Match participants rarely choose education when they enter the program. 
They are more interested in immediate employment and tend to come back to 
Project Match for education only after they have cycled through several jobs, 
according to program staff. . 

Oklahoma's Integrated Family Services (IFS) Program, which operates in nine 
counties^ assesses social service needs intensively and attempts to address a host of 
family issues, including those that affect parents' employability. Staffed by a special 
unit within the state Department of Human Services, the IFS program is designed 
for families identified as having multiple needs tliat cannot be handled simply by 
referring the family members to services. The criteria for defining a multiple-needs 
family are set by a committee in each county and reflect local service priorities. 
(Teenage mothers on AFDC are a freq?^ent target group; others include families 
affected by plant closings, families with a member fighting drug or alcohol 
dependency, and families with a medical crisis.) Families are referred for IFS 
services by multiple sources within the community. 

The initial assessment is conducted by an IFS caseworker during a home visit, 
which typically takes about 45 minutes. It results in a rep>ort on the family's 
perception of its needs and the caseworker's observations about how well the family 
is functioning. Caseworkers are encouraged to use their informed judgment in 
conducting assessments; to facilitate this, assessment protocols require descriptive 
observations rather than answers to narrowly defined questions. 

Based on the information obtained during the initial home visit, an inter-agency 
team meets to decide what interventions seem appropriate and feasible, and which 
organizations can best provide the needed services. An IFS caseworker coordinates 
the effort to resolve the family problems. Additional home visits are made as needed 
to monitor progress. By design, IFS caseloads are kept small— a maximum of 25 per 
caseworker. A case remains open until the family is able to v/ork on its problems 
without IFS help through other social service programs, or until the service goals of 
the plan have been met. 



^*See L. Olson and T. Herr, Building Opportunity for Disadvantaged Young Families: The Project Match 
Experience (Evanston, 111.: Northwestern University, Center for Urban Affairs and Policy Research, 
February 1989); and Herrand Halpem, Qmnging WImt Counts. 



67 



Chapter 8 



Operational Considerations 

The multiplicity of assessment options, and competing claims about their benefits, 
can be daunting. Not only do JOBS administrators need to make judgments about 
the likely effectiveness of different approaches to assessment, they also need to 
understand the implications of their assessment choices for the efficient operation 
of their programs, and how well Uiose choices can be carried out in the program 
environment. Often, practical considerations-such as whether JOBS participants will 
be more likely to keep one type of assessment appointment than anothei-can be 
decisive, especially when it is not clear how much different assessment approaches 
affect participants' employment and earnings or welfare receipt. 

This chapter describes such considerations, beginning with illustrations of "client 
flow" in programs discussed in the previous chapter: Assessment can speed this up 
or slow it down. Also discussed here are funding, staffing, information-management, 
and other issues. 

aient Flow in JOBS: How Many Participants 
Go Where How Quickly? 

Getting participants to "where tiiey are going" in JOBS is not simple, and pre- 
JOBS programs offer lessons about how assessment affects client flow. 

1. The experience of programs that have used a relatively extensive assessment 
as their initial sorting mechanism suggests that this practice may delay or reduce 
entry into emplojmnent-related components: 

• In the Pennsylvania Saturation Work Program (PSWP), in which tiie initial 
assessment was designed to serve as a quick screen and sorter, backlogs of 6 to 
12 weeks in scheduling orientations and appraisals led staff to eliminate 
assessment for individuals who had a high school diploma or GED, or recent 
work experience and current work skills. 

• New Jersey REACH staff reported that about 50 percent of the clients referred to 
an outside agency for in-depth assessment did not show up for their first 
appointment, slowing their progress through the assessment process. 

• In California, most of the eight counties included in MDRC's early implementa- 
tion study of GAIN were able to move participants through the initial appraisal 
within three weeks of orientation. But in one county-which emphasized personal 
counseling and had a high staff-client caseload ratio-the initial appraisal 
interview did not take place until three months after GAIN orientation. Also, 
while vocational assessments in most GAIN locations required only between 6 
and 12 hours of interviews and tests, it typically took participants two to four 
days to complete the process, and sometimes as long as two to four weeks. 
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♦ The in-depth assessment provided by one JTPA agency for participants in 
Maine's ASPIRE program is scheduled to last two to three weeks, but staff report 
it frequently takes a month to complete. 

♦ Wider Opportunities for Women, a voluntary program serving a population 
similar to that targeted in JOBS, reports that only 50 percent of those who start 
the lengthy assessment process complete it and enter an education or training 
program. 

2. Because of normal welfare caseload dynamics, JOBS programs will 
experience considerable declines in participation in the activities that occur 
second, third, and fourth in a sequence, regardless of the nature of the 
components.^^ Furthermore, the longer it takes for participants to get to their first 
activity, the fewer will participate. Some will have left welfare on their own; some 
who were available to attend an initial orientation will have developed problems 
with child care, transportation, or health; some will have moved; and some will 
have lost interest. On the other hand, assessment approaches that succeed in 
building commitment and providing needed supports may overcome some of this 
"natural" tendency for participation to decline over time. 

3. Early literacy testing probably leads to a higher rate of assignment to 
education than literacy testing at a later stage of a program. JOBS regulations 
define basic literacy as functioning at an 8,9-grade level and specify education for 
participants who are not proficient at this level (unless they have completed high 
school or its equivalent or have long-term employment goals that do not require a 
high school education). Tliese regulations pose a dilemma for states and localities 
implementing JOBS programs intended to emphasize immediate employment: 
Should the basic literacy skills of all participants be measured using standardized 
tests as a first step, or should testing take place after an initial program activity? 
Because the sequence of activity options offered to participants, and the way staff 
present these options, can have a profound influence on what participants choose 
for themselves, this is a key assessment design choice. When staff and participants 
have literacy test results in hand indicating basic skills below the ninth-grade level, 
they may believe that basic skills remediation is their only option. But individuals 
with low literacy skills can (and often prefer to) find jobs, especially where the 
AFDC grant amount for most recipients is less than local entry-level wages. 

Resources for Assessment 

Administrators of JOBS programs that offer multiple activity options at an early 
stage face a different problem: Is it more efficient to try to perfect the match 
between participants and services by conducting very comprehensive and intensive 
assessments, or to guide choices with less formal assessment and tolerate changes 
in assignments when matches do not work out? Another key question is whether, 
for philosophical reasons, uniform assessment strategies (as opposed to targeted or 



^%€e Gueron and Pauly, From Welfare to Work. 
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staged strategies) are desirable. In general, the scale of a JOBS program and the 
resources available for assessment are most critical in answering these questions. 
Many programs will not be able to afford comprehensive and intensive assessments 
for all participants. To conserve resources, policymakers in states and localities 
operating multiple-option JOBS programs may need to choose strategies that 
match low-cost assessment approaches to low-cost services and high-cost 
assessment approaches to high-cost services. 

MDRCs GAIN implementation research found that staff in counties with low 
staff-client ratios-i.e., limited human resources— were more likely to process 
assessments routinely, to make assignment decisions as though they were 
prescribed, even when participants were allowed a choice, and to accept 
participants' career preferences without in-depth exploration.^ If the philosophy 
of a JOBS program is to encourage exploration and client participation in 
assignment decisions, the resources to make this possible must be provided, for 
example, by maintaining high staff-client ratios in JOBS or contracting for this 
assessment service with other organizations. To illustrate: Twice as many clients 
can be moved through an initial intake process in a day if interviews last a half hour 
instead of an hour; twice as many staff members are needed to handle the same 
number of clients if interviews take one hour instead of a half hour. Similarly, an 
assessment of basic literacy that uses a 21^-hour test will require more staff or more 
time than an assessment that uses a one-hour test. 

If resources for assessment in JOBS are limited, cost-reduction choices include: 

• Less expensive technologies (e.g., paper-and-pencil tests rather than computerized 
assessment systems); 

• Information-gathering methods that entail less staff time (e.g., those that rely on 
self-oompleted forms, group procedures, and brief one-on-one interviews rather 
than extensive testing and one-on-one counseling); 

• Strategies that take advantage of welfare caseload dynamics (people leaving 
welfare on their own) to minimize the number of participants who need to be 
assessed (e.g., by conducting in-depth or more intensive assessments at a later 
ratlier than an earlier stage in the program); 

• Strategies that take advantage of the assessment activities that occur in certain 
program components such as basic skills education; and 

• Targeting strategies (e.g., conducting more expensive assessments for participants 
who select or have passed staff saeening for more expensive services). 



'^Rlccio et al, CAIN: Early Impleineittalion Experiences and Lessons. 
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Sequencing Assessment 

As discussed in Chapter 5, vocational assessments can be time-consuming and 
expensive. Because most training providers require a minimum literacy level, the 
decision to send a participant to training can def)end on her basic literacy skills. 
Individuals who do not meet the requisite skill levels will presumably require 
education before entering training, unless JOBS program administrators can make 
special arrangements with training vendors, or local options include both education 
and training. It may therefore be sensible to defer an inKiepth vocational assessment 
until a participant has attained the necessary literacy skills to enter training. 

Vocational assessment could thus be designed as a two-stage process. An initial 
assessment can elicit enough information (on basic skills, education, job skills, work 
experience, and general vocational goals) to determine whether training is an 
appropriate first activity. If so, the individual could then go through an in-depth 
vocational assessment to determine a specific training course. If not, the in-depth 
assessment could be deferred until training is deemed appropriate. Similarly, literacy 
assessment may be structured as a two-stage process. First could come up-front 
screening to establish the basic skills levels of all participants. A later assessment 
could indicate the type of education program that participants with low basic skills 
need. Then-for those referred to education— an in-depth diagnosis to determine more 
precisely the strengths and weaknesses to be addressed might be carried out by the 
educational institution. The lengthier and more detailed test required for diagnostic 
assessment can wait until participants actually enter an education program. 

This strategy makes sense for the client as well as the program. The intervening 
time and activities might enable her to expand her horizons and refine her career 
goals, particularly if career exploration is part of the process. In addition, because 
many of the tests used in a vocational assessment are written for individuals with 
a seventh- or eighth-grade reading level, they are not appropriate for participants 
reading below that level. Used with less skilled readers, the tests may not provide 
an accurate picture of vocational aptitudes, interests, or knowledge. 

A two-tiered strategy also makes sense from the client-flow perspective. Because 
of the large proportion of participants in welfare employment programs found to 
need basic educatioit-combined with the effect of normal turnover in reducing 
participation in the later activities of sequenced programs— only a small proportion 
of those who enter JOBS programs without the basic skills required for training will 
in fact enter a training program. Early information from MDRC's evaluation of 
GAIN, for example, showed that within six months of program entry, less than 1 
f)ercent of the mandatory caseload had completed the in-depth assessment and 
entered a post-assessment education or training activity .^^ (This reflects, among 
other things, the large percentage of participants referred to education and the 
length of time it took them to complete that component.) Data from longer follow- 
up on the Baltimore Options and San Diego Saturation Work Initiative Model 
(SWIM) programs found that less than a third of the program enrollees participated 



^Riccio et al., GAIN: Early Implementation Experiences and Lessons, p. 85, Table 4.3. 
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in two or more activities^ and only a small proportion of those entered vocational 
training.^ 

A final sequencing consideration is that, in at least one instance, an up-front, 
lengtiiy, in-depth vocational assessment process has been criticized by welfare 
advocates for creating an artificial barrier to program participation. Although the 
JTPA assessment used for ASPIRE participants in one area in Maine is intended to 
increase participants' empowerment and self-knowledge, the assessment has been 
challenged as a way of delaying and denying services rather than providing them. 

Tailoring Assessment to Different Subgroups 

JOBS planners should also give thought to tailoring assessments to the specific 
needs of different subgroups of participants. Teenage mothers and women who have 
been on AFDC for many years, for example, may need in-depth social service 
assessments and vocational exploration, because they frequently face difficult family 
situations and their knowledge of the work world is often limited. As noted in 
Chapter 5, some testing instruments have been specifically designed for young 
adults. The special needs of immigrants and non-English speakers may be better 
identified if an assessment employs instruments developed specifically for such 
groups and also addresses their understanding of U.S. culture. A number of literacy 
tests (including CASAS) have versions designed for those with very limited ability 
in English. 

Staffing Issues 

Dividing Program Responsibilities 

Welfare-to-work programs have taken a variety of approaches to staffing the 
initial assessment interview. Some have separated orientation and assessment from 
other activities, creating positions in which staff perform only those functions and 
have no ongoing contact with participants. Others give responsibility for the initial 
assessment to case managers, who continue to work with clients throughout their 
participation in JOBS. Field research in MDRCs GAIN evaluation found that when 
case managers did assessments, they took a more personal interest in clients and 
probed more deeply into their circumstances and plans. However, there is no 
research evidence to show that one approach-or one staffing pattemr-lead^ to 
greater program impacts than another .^^ 

Case management may be particularly important in programs that attempt to 
address the needs of JOBS participants' families. To date, a hallmark of the few 
welfare-to-work programs that take this approach is intensive and aggressive case 



^G. Hamilton, "Participant Flow Patterns in Different Types of Welfare Employment Programs," 
unpublished MDRC docunnent. May 16, 1990. 

^^See J. Riccio and K. Sherwood, Key Decisions in Implementing JOBS: Lessons from California's GAIN 
Program (New York: MDRC, 1990). 
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management by specialized staff.^ There is no evidence to show that this is the 
most efficient or the only possible staffing arrangement for such programs, however. 

Multiple responsibilities can create problems in moving clients through a 
sequence of activities. In one welfare employment program operated in the 1980s, 
for example, the staff responsible for the initial assessment interviews were also 
responsible for conducting the job-search workshops that participants attended after 
assessment. The dual roles were found to be somewhat in conflict: The more time 
staff spent on assessments, the fewer workshops they scheduled. Backlogs 
developed at both ends. 

Many programs refer participants to outside agencies for in-depth assessments 
of their readiness for vocational training and recommendations for specific training 
courses, whether such assessments are done initially or later. Typically, the welfare 
agency contracts with another organization— most commonly, the local JTPA agency 
or a community college— to conduct vocational assessments of all the clients it refers, 
and to recommend appropriate occupations and specific training or education 
courses in a written report. In some states, staff from the two agencies meet to 
discuss individual cases. After the report is completed, the JOBS participant 
discusses it with a JOBS staff member, and the welfare agency prepares an 
employability plan. 

Some states simply contract for whatever assessment is usually done by the 
outside agency. Others set standards for the assessment process and instruments, 
and do not pay for any testing that does not meet those standards, or they negotiate 
for specific assessment practices and procedures. As mentioned, the amount of 
money and the number of clients likely to flow from the JOBS program to these 
other organizations often give the welfare agency considerable leverage. 

The rationale for contracting out the vocational assessment is that the education 
and training agencies have more expertise in such matters than the welfare agency 
and have already made the necessary capital investment. In addition, it can help 
prevent situations in which welfare recipients complete an in-depth assessment at 
the welfare agency only to go through a similar evaluation process at a service 
provider agency. 

Disadvantages of such arrangements are that the welfare agency can lose control 
over the assessment process and that the philosophical approach of the other agency 
may not reflect that of the JOBS program. In addition, some staff in both California 
and New Jersey reported that if the agency doing the assessment is itself a service 
provider, the assessments tend to recommend that JOBS participants enroll in the 
agency's services. JOBS staff also voiced concern that provider agencies often use 
assessments to screen people in or out of specific program offerings, rather than to 
find the most appropriate service among a range of providers. Referrals to outside 
agencies for assessment also require careful monitoring to track the progress of 
participants, particularly if the assessment takes several days or weeks to complete. 



*^See Golden et al, Welfare Rsform and Poor Children, 
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Staff Qualifications 

Here, too, program experience is varied and there is little consensus. Upgraded 
eligibility workers, case managers with social service backgrounds, and employment 
specialists have all been used to do appraisals and assessments in work/welfare 
programs. Programs tend to hire people with more than a high school education for 
such positions, but this is not always the case. Nor do states always require that 
staff be college graduates. The regulations governing California's GAIN program are 
quite explicit in specifying the qualifications of staff who conduct in-depth 
vocational assessments— either graduation from an accredited college or two years 
of counseling experience— but other states have not established hiring guidelines for 
their subcontractors.^^ 

When asked what qualifications are necessary, program administrators note the 
importance of other criteria in addition to formal education or prior experience. 
Some stress the need for staff v^th good "people skills" who can draw clients out 
about their goals and circumstances. Experts and practitioners interviewed for this 
paper stressed that an assessment staff needs a thorough knowledge of the 
vocational training and education offerings in the area and the characteristics of the 
institutions that offer them, as well as an awareness of outside sources that could 
help meet participants' supportive-service needs. They also mentioned the 
importance of personal networking to develop that knowledge. 

On-the-job training is important for staff responsible for assessments, particularly 
if they have not previously performed them. Many programs offer training to those 
who are new to the role, but several states mentioned the need for ongoing training 
in both assessments and assignments. (Several commercial testing systems, including 
CASAS and VITAS, provide training for the staff who will use their instruments.) 

How Assessment Staff Interpret Program Goals 

The attitudes and philosophy of assessment staff will influence the choice of 
activities and thus affect the flow of participants and the use of resources. In PSWP 
in Philadelphia, for example, although the program objective was to move 
participants quickly into employment, staff started to refer individuals who did 
poorly on a basic skills test into basic education classes, because they felt the test 
results indicated that the participants needed education in order to be employable. 
In some GAIN counties, also, staff tended to steer individuals into education 
programs rather than job search when the program model called for a choice. In 
Maine, up-front, in-depth vocational assessment that emphasizes career exploration 
and preparation for employment leading to self-sufficiency instead of quick job 
placement has tended to route participants into long-term training and education 
activities. As a result, there are not always sufficient resources to meet the needs of 
new enrollees. 

Staff training and administrative oversight are essential to ensure that assessment 
procedures are either carried out as intended or iuteritiorially modified. Involving 



^'Sec Riccio et al., GAIN: Early Jmpleiiientation Experiences and Lessons, p. 220. 
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program staff in the design of assessment approaches is another way to improve 
consistency between their decisions and administrators' goals. 

Keeping Track of Information 

To avoid duplication of effort, JOBS planners need to ensure that information 
collected at one point in the process is passed on to staff at later stages. This can be 
hard to do when some assessment functions are performed by outside agencies or 
staff. JOBS staff members who conduct assessments should be given all the 
information derived from in-house assessments, including work history, education, 
employment goals and interests, test results, and social service needs. They will also 
need to know the findings of assessments conducted by an outside agency, and vice 
versa. Devising procedures to be certain that information flows smoothly in both 
directions is worth the trouble. 

Ideally, each step in the assessment process should build on or add to the 
information previously collected; it is wasteful of staff resources and tedious for 
participants if different stages in the assessment process repeatedly elicit the same 
information. JOBS programs will therefore need a record-keeping system that 
updates and adds information on the participant and her JOBS activities throughout 
her participation in the program, and makes this information readily available to 
staff. To ensure that the records will be reviewed and used at subsequent stages in 
the process, it may be useful to have them routinely provided to the staff 
responsible for different parts of an assessment, rather than simply making them 
available upon request. 
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Program Design and Assessment 
Trade-offs 

By intrcducing new incentives for investing in human capital and serving more 
welfare recipients in employment programs, JOBS puts a premium on the ability of 
program operators to match participants to services that will improve their job 
prospects. But the best methods for doing this are still in doubt-in part because the 
available evidence does not permit accurate predictions of what it takes to succeed 
in the job market, and in part because definitions of success vary. Assessment 
technologies and strategies can help pinpoint a variety of participant characteristics 
that suggest paths to employment and, perhaps, self-sufficiency, but they do not 
promise either a precise plan or a sure outcome. Thus, in the same way tiiat tiie 
basic structure of JOBS presents choices to program planners, designing an 
assessment system for JOBS entails trade-offs. The premise of tiiis paper is tiiat, to 
the extent that previous experience and research can help, JOBS planners need to 
know what they are trading off when they choose among assessment approaches. 

An important potential trade-off, but one that cannot be precisely quantified, is 
between short-term and long-term gains for program participants. Underlying the 
assessment function-and central to the innovations in JOBS-is the notion that some 
groups of participants will not nnake either short-term or long-term gains in earnings 
without some type of education or training. Also embedded in many assessment 
approaches is the idea that some individuals who could make short-term gains will 
do better in the long run if their basic literacy skills or vocational skills are 
upgraded through education or training. (Often, the corollary is that skills 
upgrading cannot take place on the job.) 

Should assessment be geared toward short-term or long-term gains? This 
question can be answered only in the context of JOBS program goals, which 
assessment approaches reflect. One lesson that some pre-JOBS program operators 
learned is that formal assessment approache&-i.e., those that employ standardized 
tests-tend to tip programs more toward education and training and less toward 
immediate employment. There are nnany reasons for this: Formal assessment 
procedures uncover and highlight the generally poor basic skills of the welfare 
population; staff who conduct these assessments are often philosophically inclined 
toward human capital investments (especially if they work for education or 
vocational training providers); assessment results may influence participants' 
preferences by raising their awareness of their own needs for education and 
training; and front-line staff in welfare employment programs draw conclusions 
about what works based on their direct experience vrith clients, which is often 
concentrated among individuals who return to welfare after spells of employment 
that lead neither to skills improvement nor to earnings sufficient to support their 
families. 

On the other hand, formal assessment approaches do not necessarily defer 
employment. They can be designed to balance program goals and participants' 
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needs. Choosing human capital investments in the form of education and training 
over immediate employment, with the objective of improving long-term gains for 
welfare recipients, may be the focus of a particular JOBS program and is certainly 
encouraged by the legislation. However, state and locai JOBS administrators 
should be aware that if they want to stress immediate employment for some 
participants, comprehensive and intensive up-front assessment procedures— tmless 
they are carefully designed and targeted— may work against this short-term goal. 

At the heart of the decisions that state and local JOBS administrators have to 
make about designing an assessment system in JOBS are three thorny issues: (1) 
Solid evidence on the effectiveness of assessment in employment and training 
programs, both generally and for particular approaches, is sketchy; (2) assessment 
is a program expense that, in some localities, will reduce the funds available for 
employment-related services, so investing wisely in this service is doubly important; 
and (3) there are similar trade-offs in deciding about assessment approaches as in 
deciding about other program services. When resources are limited, programs can 
provide a little assessment for everyone, a lot of assessment for a small group, or 
targeted assessment for participants for whom this service is most needed or most 
likely to pay off in better "matches" with employment services. 
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